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EXECUTIVE  SUMMARY 


PROBLEM 

The  Navy,  DoD,  and  private  industry  need  new,  innovative  and  effective  instructional 
methods  and  media  to  ensure  that  personnel  have  the  necessary  skills  to  perform  in  today’s  high 
technology,  rapidly  changing  environment.  There  is  a  growing  advocacy  for  the  use  of 
instructional  games  as  a  primary  delivery  medium  for  this  required  instruction.  However,  the 
decision  to  use  instructional  games  is  often  made  on  the  basis  of  “leaps  of  faith”  rather  than 
empirical  data  on  the  effectiveness  of  instructional  games. 

OBJECTIVE 

The  objective  of  this  effort  was  twofold: 

•  To  conduct  a  review  of  the  literature  on  instructional  games  with  a  focus  on  the  empirical 
research  on  the  instructional  effectiveness  of  games. 

•  To  document  conclusions  and  recommendations  from  this  literature  review. 

APPROACH 

This  effort  proceeded  through  the  following  steps: 

•  Researchers  in  the  field  of  instructional  games  were  contacted  to  obtain  literature  reviews 
either  published  or  in  progress. 

•  The  Internet  was  searched  to  obtain  additional  documents. 

•  Review  articles  were  read  to  identify  empirical  research  articles. 

•  Articles  that  documented  empirical  data  on  the  instructional  effectiveness  of  games  were 
obtained. 

•  The  empirical  articles  were  read  and  summarized. 

•  The  empirical  data  were  used  to  draw  conclusions  and  recommendations  about  the  design 
and  use  of  games  as  instructional  media. 

RESULTS 

Two  hundred  and  seventy-four  documents  on  the  design,  use,  and  evaluation  of  games  were 
reviewed.  Of  these,  169  could  not  be  used: 

•  77  of  these  articles  were  not  used  because  they  only  provided  the  author’s  opinion  on  the 
potential  of  instructional  games 

•  1 1  articles  were  on  game  design  and  analysis 

•  29  articles  were  simulation  studies  (not  games) 

•  14  articles  were  computer-based  instruction  studies  (not  games) 

•  36  articles  documented  studies  with  major  methodological  problems,  and 

•  2  were  proposals  for  future  research. 

One  hundred  and  five  articles  are  documented  in  this  report.  These  included: 

•  26  review  articles 


5 


Technical  Report  2005-004 


•  3 1  theoretical  articles,  and 

•  48  articles  that  provided  empirical  data  on  the  effectiveness  of  instructional  games. 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  review  of  empirical  research  on  the  effectiveness  of  instructional  games  leads  to  the 
following  five  conclusions  and  four  recommendations.  The  conclusions  are: 

•  The  empirical  research  on  the  effectiveness  of  instructional  games  is  fragmented.  The 
literature  includes  research  on  different  tasks,  age  groups,  and  types  of  games.  The  research 
literature  is  also  filled  with  ill  defined  terms,  and  plagued  with  methodological  flaws. 

•  Although  research  has  shown  that  some  games  can  provide  effective  learning  for  a  variety  of 
learners  for  several  different  tasks  (e.g.,  math,  attitudes,  electronics,  and  economics),  this  does 
not  tell  us  whether  to  use  a  game  for  our  specific  instructional  task.  We  should  not  generalize 
from  research  on  the  effectiveness  of  one  game  in  one  learning  area  for  one  group  of  learners 
to  all  games  in  all  learning  areas  for  all  learners. 

•  There  is  no  evidence  to  indicate  that  games  are  the  preferred  instructional  method  in  all 
situations. 

•  Instructional  games  should  be  embedded  in  instructional  programs  that  include  debriefing 
and  feedback  so  the  learners  understand  what  happened  in  the  game  and  how  these  events 
support  the  instructional  objectives. 

•  Instructional  support  to  help  learners  understand  how  to  use  the  game  increases  the 
instructional  effectiveness  of  the  gaming  experience  by  allowing  learners  to  focus  on  the 
instructional  information  rather  than  the  requirements  of  the  game. 

The  following  four  recommendations  may  help  the  instructional  gaming  industry  produce 
more  instructionally  effective  games. 

•  The  decision  to  use  a  game  should  be  based  on  a  detailed  analysis  of  the  learning 
requirements  and  an  analysis  of  the  tradeoffs  among  alternate  instructional  approaches. 

•  Program  managers  and  procurement  personnel  should  insist  that  game  developers  clearly 
demonstrate  how  the  design  of  a  game  will  provide  interactive  experiences  that  support 
properly  designed  instructional  objectives  (see  for  example,  Gagne  &  Briggs,  1979;  Merrill, 
1983;  1997  for  guidance  on  the  proper  design  of  instructional  objectives). 

•  Instructors  should  view  instructional  games  as  adjuncts  and  aids  to  help  support  instructional 
objectives.  Learners  should  be  provided  with  debriefing  and  feedback  that  clearly  explains 
how  their  experiences  with  the  game  help  them  meet  these  instructional  objectives. 

•  Instructor-less  approaches  (e.g.,  web-based  instruction)  must  include  all  “instructor 
functions.”  These  include  performance  evaluation,  debriefing,  and  feedback. 
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INTRODUCTION 


PROBLEM 

Recently  there  has  been  increased  interest  in  the  use  of  instructional  games.  It  has  been 
argued  that  young  people,  raised  playing  video  games,  have  changed  in  ways  that  turn  them  off 
to  conventional  instruction  (Prensky,  2001,  p.  01-6).  It  has  also  been  suggested  that  instructional 
games,  because  of  their  applied  and  dynamic  nature,  can  heighten  the  learner’s  motivation  and 
interest  more  effectively  than  classroom  lectures  (Greenblat,  1981,  p.  147).  Another  suggested 
benefit  of  instructional  games  is  that  they  may  improve  the  retention  of  learned  skills  and 
knowledge  (Pierfy,  1977). 

Unfortunately,  many  decisions  about  whether  to  use  games  for  instruction  are  based  on 
unfounded  assumptions  about  the  ability  of  games  to  provide  effective  and  efficient  instruction. 
This  problem  was  stated  by  a  senior  naval  officer  as  follows.  “Lack  of  quantifiable  metrics  or 
data  to  validate  use  of  gaming  technology  as  an  effective  delivery  media  [s/c]  for  various  training 
solutions  has  relegated  decisions  of  the  same  to  being  ‘leaps  of  faith’  vice  based  on  solid 
business  case  solutions”  (M.  K.  Gritton,  personal  communication,  September  26,  2005). 

OBJECTIVE 

The  objective  of  this  report  is  to  provide  a  review  and  discussion  of  empirical  research  on  the 
effectiveness  of  instructional  games.  Based  on  this  review,  a  series  of  conclusions  and 
recommendations  are  provided. 

ORGANIZATION  OF  THE  REPORT 

This  report  contains  six  sections.  Following  the  introduction,  the  second  section  summarizes 
a  variety  of  descriptions  and  classifications  of  games.  The  third  section  discusses  how  games  are 
not  always  used  for  instructional  purposes,  but  how  the  characteristics  of  games  (e.g.,  challenge, 
rules,  and  context)  can  motivate  learners.  The  fourth  section  provides  summaries  of  review 
articles  on  the  effectiveness  of  various  games  and  empirical  articles  on  the  effectiveness  of 
specific  games.  The  fifth  section  summarizes  the  empirical  research  and  compares  what  we 
know  about  instructional  games  today  with  several  “claims”  made  about  instructional  games 
twenty- five  years  ago.  Finally,  a  series  of  conclusions  and  recommendations  are  provided  in  the 
final  section,  followed  by  a  complete  reference  list. 

CONFUSION  IN  TERMS 

Certainly,  it  can  be  argued  that  games  can  attract  and  hold  an  individual’s  attention.  It  is  less 
certain  that  they  always  provide  instructional  value.  Before  discussing  the  research  on 
instructional  games,  it  is  important  to  distinguish  games  from  other  instructional  activities.  In 
the  literature  on  instructional  games,  we  often  find  the  terms  simulations,  games,  simulation- 
games,  and  computer  games  used  interchangeably  (e.g.,  Greenblat  &  Duke,  1981;  Reiber,  1996; 
Thomas,  Cahill  &  Santilli,  1997).  Greenblat  (1981)  observed  that  “in  many  studies,  ‘games’  and 
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‘simulations’  are  at  least  implicitly  treated  as  homogeneous”  (p.  181).  Although  aware  of  the 
problem,  even  Greenblat  used  the  term  simulation  in  one  sentence  and  the  term  game  in  the  next 
sentence  to  describe  the  same  thing  (p.  144). 

All  simulations  are  based  on  models  of  reality.  A  model  can  be  defined  as  “a  physical, 
mathematical,  or  otherwise  logical  representation  of  a  system,  entity,  phenomenon,  or  process 
(Department  of  Defense,  1997,  p.  138).  A  simulation  is  “a  method  for  implementing  a  model 
over  time”  (p.  160).  Thus,  a  model  provides  the  rules  and  data  that  are  used  to  represent  some 
portion  of  reality.  Then,  a  simulation  uses  these  rules  and  data  to  dynamically  represent  these 
aspects  of  reality  for  some  purpose.  A  specific  type  of  simulation  is  sometimes  called  a 
microworld  (e.g.,  Miller,  Lehman,  &  Koedinger,  1999).  Microworlds  are  simulations  that 
attempt  to  capture  the  relevant  aspects  of  some  topic  or  phenomena  so  learners  can  interact 
within  it  to  observe  the  effects  of  their  interactions.  For  our  purposes,  micro  worlds  will  be 
referred  to  as  simulations  unless  there  is  a  reason  to  make  a  distinction  between  the  two  terms. 
Simulations  can  incorporate  aspects  of  games  (e.g.,  competition,  rules,  rewards,  etc.)  or  be  used 
as  games,  but  only  if  the  simulation  includes  these  game  aspects,  should  it  be  called  a  simulation 
game. 

One  of  the  purposes  of  simulations  can  be  to  provide  instruction,  but  only  if  they  are  designed 
to  do  so.  For  example,  Simons  (1993)  believes  that  “the  educational  value  of  a  simulation  comes 
with  repeated  simulation  of  a  model”  (p.  137).  In  this  type  of  application,  the  learner  is  able  to 
try  various  policies  or  parameters  and  observe  the  results.  Through  this  closed-loop  process,  the 
learner  gradually  builds  an  understanding  of  the  simulated  system.  This  is  a  powerful 
instructional  benefit  of  simulation  games.  Unfortunately,  most  of  these  games  do  not  meet  their 
potential.  Simons  explains  that  simulation  games  are  ineffective  because  they  do  not  “directly 
communicate  their  underlying  models  and  the  reasons  for  those  models’  behaviors”  (p.  148). 

One  might  rephrase  this  in  instructional  terms — the  simulation  game  is  ineffective  because  it 
does  not  directly  address  instructional  objectives  and  does  not  ensure  that  the  learner  has  met 
those  objectives. 

Not  every  game  is  intended  to  be  used  for  instruction.  Most  games  are  intended  for 
enjoyment.  Just  putting  a  game  in  a  course  does  not  ensure  that  it  will  facilitate  the  learning  of 
the  intended  knowledge  or  skill.  Games  can  be  designed  for  instruction,  but  only  if  they  are 
designed  to  support  specific  instructional  objectives  and  are  incorporated  logically  into  an 
instructional  program.  Before  elaborating  on  this  point,  it  is  useful  to  discuss  the  various  types 
of  games  and  the  characteristics  that  make  them  games. 
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DESCRIPTIONS  AND  CLASSIFICATIONS  OF  GAMES 

Some  authors  (e.g.,  de  Felix  &  Johnson,  1993)  described  games  by  listing  their  structural 
components,  such  as  dynamic  visuals,  interaction,  rules,  and  goals.  Others  (e.g.,  Gredler,  1996) 
stated  that  the  essential  elements  of  a  game  are  the  task,  the  player’s  role,  the  multiple  paths  to 
the  goal,  and  the  degree  of  player  control.  Baranauskas,  Neto,  and  Borges  (1999)  stated  that  the 
essence  of  gaming  is  challenge  and  risk. 

Csikszentmihalyi  (1990)  discussed  how  Roger  Caillois  (1958/2001),  the  French 
psychological  anthropologist,  classified  games  into  four  broad  classes  based  on  the  kind  of 
experiences  they  provide. 

1.  Agnostic  games  are  those  that  have  competition  as  their  main  feature  (e.g.,  sports  and 
athletic  events).  These  types  of  games  stretch  the  player’s  skills  to  meet  the  challenge 
provided  by  the  skills  of  their  opponents. 

2.  Aleatory  games  are  games  that  involve  the  element  of  chance  (e.g.,  dice  or  bingo).  These 
games  give  the  illusion  that  the  player  somehow  controls  the  inscrutable  future. 

3.  Vertigo  or  Ilinix  games  are  activities  that  alter  one’s  consciousness  by  scrambling 
ordinary  perception.  Examples  are  riding  a  merry-go-round,  skydiving,  or  young 
children  turning  around  in  circles  until  they  are  dizzy. 

4.  Mimicry  games  allow  the  player  to  create  alternative  realities  (e.g.,  dance,  theater,  and 
the  arts  in  general).  These  games  make  one  feel  as  though  they  are  more  than  they 
actually  are  through  fantasy,  pretence,  and  disguise.  Caillois  (1958/2001)  uses  the  term 
simulation  as  a  synonym  for  mimicry  (p.  36). 

Any  given  game  may  include  some  or  all  of  these  experiences,  but  it  is  usually  designed  to 
only  provide  a  subset  of  these  experiences.  Furthermore,  games  are  played  with  a  variety  of 
media,  from  game  boards  to  networked  computers.  It  is  not  the  medium  on  which  the  game  is 
played,  but  its  characteristics  that  make  it  a  game.  One  of  the  most  important  of  these 
characteristics  is  the  specificity  of  the  game’s  rules.  Caillois  (1958/2001)  discussed  the  issue  of 
rules  by  defining  a  continuum  anchored  by  the  terms  paidia  and  ludus.  He  defined  paidia  as 
“the  spontaneous  manifestations  of  the  play  instinct”  (pp.  27-28).  Games  at  the  paidia  end  of  the 
continuum  have  few  or  no  rules  and  are  played  for  shear  joy.  At  the  other  end  of  the  continuum 
we  find  ludus,  which  “is  complementary  to  and  a  refinement  of  paidia,  which  it  disciplines  and 
enriches”  (p.  29).  Ludus  refers  to  games  with  rules  and  requirements  for  play.  The  more  a  game 
is  bounded  by  specific  rules,  the  closer  it  falls  toward  the  ludus  end  of  the  continuum.  For 
example  the  paidia  feelings  found  in  illinix  (vertigo)  games  are  challenged  and  constrained  by 
the  rules  and  skills  required  in  mountain  climbing  or  tightrope  walking. 

The  “Folk  Model”  divides  games  into  4  (non-exclusive)  categories:  games  of  skill,  games  of 
chance,  games  of  strategy,  and  simulation  games  (Wikipedia,  2005). 

1.  Games  of  skill  include:  board  games,  card  games,  letter  games,  mathematical  games, 
puzzle  games,  guessing  games,  word  games,  games  of  physical  skill,  and  instructional 
games. 

2.  Games  of  chance  include:  dice  games,  card  games,  casino  games,  lottery-type  games, 
Bingo,  and  Pinata. 
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3.  Games  of  strategy  include:  Checkers,  Chess,  Go,  and  Mastermind. 

4.  Simulation  games  include:  role-playing  games,  board  games  like  Monopoly,  and 
computer  and  video  games. 

Bjork  and  Holopainen  (2003)  describe  games  in  terms  of  four  overlapping  conceptual  groups. 

1 .  The  overall  activity  of  the  game.  This  is  how  the  players  understand  the  meaning  of  the 
activity  and  how  the  activity  unfolds.  Sub  components  of  activities  include: 

•  The  game  instance,  which  is  the  specific  players,  their  experience,  and  the  location 
and  requirements  of  a  single  completion  of  a  game. 

•  The  game  session,  which  is  the  activity  defined  by  the  time  spent  on  playing  a 
game  instance. 

2.  The  boundary  components  of  the  game.  These  include  the  rules,  modes  of  play,  and 
goals  of  the  game. 

3.  The  temporal  components  of  the  game.  These  are  used  to  record  the  activity  of  playing 
the  game.  They  include  discrete  actions  (e.g.,  moving  a  chess  piece)  or  continuous 
actions  (e.g.,  driving  a  racing  car).  Temporal  components  include: 

•  Events,  which  are  discrete  points  in  the  game  where  the  game  state  changes. 

Events  usually  occur  as  the  result  of  player  actions,  but  can  also  be  the  result  of 
trigger  events  such  as  elapsed  time. 

•  Closures  are  a  change  of  game  state  resulting  from  completion  of  a  goal  or  subgoal. 

•  End  conditions,  which  specify  when  a  closure  occurs. 

•  Evaluation  functions,  which  determine  the  outcome  of  an  event. 

4.  The  objective  components  of  the  game,  such  as  the  types  of  players,  the  game  interface, 
and  the  physical  and  logical  components  that  inform  players  about  the  current  game  state 
(e.g.,  tokens  that  represent  which  player  is  currently  in  control  or  physical  elements  that 
describe  the  game  space  like  chess  squares). 

Leemkvil,  de  Jong,  and  Ootes  (2000)  and  Bright  and  Harvey  (1984)  discussed  several 
characteristics  of  games.  According  to  these  authors,  all  games  include: 

•  Voluntary  Play.  A  game  is  freely  engaged  in.  Persons  normally  play  games  because  they 
want  to,  not  because  it  is  required. 

•  Some  goal  state  that  must  be  reached.  The  goal  state  may  be  the  same  each  time  a  game 
is  played  or  it  can  change  over  time.  The  goal  may  also  consist  of  subgoals  that  may 
change  during  play  (e.g.,  when  a  player  reaches  a  certain  proficiency  and  moves  to  a 
higher  difficulty  level).  Players  can  sometimes  set  their  own  goals. 

•  Constraints  and  rules.  These  define  which  actions  are  allowed  and  which  are  not.  They 
also  define  the  setting  and  goals  of  the  game.  Certain  actions  may  introduce  additional 
constraints,  often  defined  by  if-then  statements  (e.g.,  If  a  player  misses  a  target  three 
times,  then  he  or  she  must  complete  a  practice  session  and  achieve  a  criterion  score  before 
the  game  can  continue).  Constraints  are  sometimes  induced  by  setting  time  limits  on 
certain  actions. 

•  Competition.  Players  can  compete  in  several  ways: 

o  With  other  players  or  teams  by  beating  them  to  a  goal  or  outperforming  them  by 
achieving  a  higher  score. 

o  With  the  system  (e.g.,  by  moving  to  higher  difficulty  levels). 
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o  With  themselves  (e.g.,  by  improving  their  performance). 

•  A  specific  context.  A  game  is  always  situated  in  a  specific  context  that  makes  it  more  or 
less  realistic,  appealing,  or  motivating  to  the  players.  This  context  is  separate  from  real 
life  in  terms  of  time  and  space.  Leemkvil  et  al.  quote  Graham  and  Grey  concerning  how 
game  players  must  place  themselves  into  the  situated  game  context.  “In  one  sense  all 
gaming  involves  role  playing  since  the  individual  participants  are  asked  to  assume  the 
situation  assigned”  (Leemkvil,  et  al.,  2000,  p.  15  citing  Graham  and  Grey). 

Dorn  (1989)  defines  a  game  as  “any  contest  or  play  among  adversaries  or  players  operating 
under  constraints  or  rules  for  an  objective  goal”  (p.  2).  He  defines  a  simulation  as  “an  operating 
representation  of  central  features  of  reality”  (p.  2).  He  then  defines  a  simulation  game  as 
“activities  undertaken  by  players  whose  actions  are  constrained  by  a  set  of  explicit  rules 
particular  to  that  game  and  by  a  predetermined  end  point.  The  elements  of  the  game  constitute  a 
more  or  less  accurate  representation  or  model  of  some  external  reality  with  which  players 
interact  by  playing  roles  in  much  the  same  way  as  they  would  interact  with  reality  itself’  (p.  3). 

A  Game  Designer’s  Definition  of  a  Game.  Greg  Costikyan  is  a  game  designer  and 
commentator  on  gaming.  He  defines  a  game  as  “an  interactive  structure  of  endogenous  meaning 
that  requires  players  to  struggle  toward  a  goal”  (Costikyan,  2002,  p.  22).  Let  us  look  at  each  of 
these  components  in  greater  detail.  Costikyan  maintains  that  a  game’s  structure  is  the  rules, 
components,  and  other  characteristics  (e.g.,  delivery  medium)  that  define  the  way  people  play 
the  game.  He  states  that  this  structure  is  interactive  because  the  game  changes  based  upon  the 
actions  of  the  players.  This  structure  also  provides  the  meaning  of  the  game.  Thus,  a  game’s 
meaning  is  endogenous — it  is  created  by  the  internal  structure  of  the  game  and  has  no  meaning 
outside  of  the  game.  Furthermore,  the  game’s  endogenous  meaning  defines  the  goals  which 
game  players  attempt  to  achieve.  Finally,  players  must  struggle  to  achieve  the  goals  of  a  game. 
In  this  sense,  struggle  means  that  the  players  must  work  to  achieve  the  goals.  A  good  game 
challenges  the  players,  often  allowing  them  to  move  to  greater  levels  of  difficulty  as  they 
become  more  skilled  at  the  game. 


HYBRID  GAMES 

As  discussed  above,  the  terms  game  and  simulation  are  often  confused.  Leemkvil,  et  al. 
(2000)  discuss  the  relationships  among  games,  simulations,  and  case  studies.  Figure  1  illustrates 
these  relationships.  As  defined  above,  a  simulation  is  “a  method  for  implementing  a  model  over 
time”  (Department  of  Defense,  1997,  p.  138).  A  case  study  is  an  actual  or  hypothetical  problem 
situation  taken  from  the  real  world.  As  illustrated  in  Figure  1,  games,  simulations,  or  case 
studies  can  be  found  in  a  pure  form  or  as  hybrids  with  the  others.  In  the  literature  on  games,  the 
simulation  game  is  the  most  often  encountered  hybrid. 
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Figure  1: 

Relationship  among  Games,  Simulations  and  Case  Studies 
ELEMENTS  OF  ENJOYMENT 

One  of  the  aspects  that  attract  people  to  games  is  that  they  are  enjoyable.  Csikszetmihalyi 
(1990)  has  studied  many  enjoyable  activities,  including  games.  He  argued  that  there  are  several 
elements  (components)  that  make  an  activity  enjoyable. 

•  A  challenging  activity  that  requires  skills.  Individuals  report  that  they  enjoy  activities 
that  challenge  their  skills,  “...the  overwhelming  proportion  of  optimal  experiences  are  reported 
to  occur  within  sequences  of  activities  that  are  goal-directed  and  bounded  by  rules— activities 
that  require  the  investment  of  psychic  energy,  and  that  could  not  be  done  without  the 
appropriate  skills”  (p.  49).  Csikszetmihalyi  goes  on  to  explain  that  “an  ‘activity’  need  not  be 
active  in  the  physical  sense  and  the  ‘skill’  necessary  to  engage  in  it  need  not  be  a  physical 
skill”  (p.  49).  What  is  necessary  is  that  the  individual  directs  his  or  her  attention  (psychic 
energy)  to  the  task  and  applies  skills  to  accomplish  it.  One  of  the  aspects  of  games  that  offer 
challenge  is  competition.  However,  “competition  is  enjoyable  when  it  is  a  means  to  perfect 
one’s  skills;  when  it  becomes  an  end  in  itself,  it  ceases  to  be  fun”  (p.  50). 

•  The  merging  of  action  and  awareness.  Another  element  of  enjoyment  is  when  one 
“loses”  oneself  in  the  activity.  “People  become  so  involved  in  what  they  are  doing  that  the 
activity  becomes  spontaneous,  almost  automatic;  they  stop  being  aware  of  themselves  as 
separate  from  the  actions  they  are  performing”  (p.  53).  This  merging  of  an  individual’s  action 
and  activity  is  why  Csikszetmihalyi  calls  an  optimal  experience  “flow.”  “The  short  and  simple 
word  describes  well  the  sense  of  seemingly  effortless  movement”  (p.  54). 
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•  Clear  goals  and  feedback.  “The  reason  it  is  possible  to  achieve  such  complete 
involvement  in  a  flow  experience  is  that  goals  are  usually  clear,  and  feedback  immediate”  (p. 
54).  This  is  another  important  feature  of  games.  It  is  usually  clear  if  a  goal  has  been  made  or  a 
target  hit.  One  does  not  have  to  wonder  if  they  performed  correctly,  the  game  provides 
immediate  feedback. 

•  Paradox  of  control.  Enjoyable  experiences  involve  a  sense  of  control — or,  “more 
precisely,  as  lacking  the  sense  of  worry  about  losing  control  that  is  typical  in  many  situations  of 
normal  life”  (p.  59).  “What  people  enjoy  is  not  the  sense  of  being  in  control,  but  the  sense  of 
exercising  control  in  difficult  situations. ..Only  when  a  doubtful  outcome  is  at  stake,  and  one  is 
able  to  influence  that  outcome,  can  a  person  really  know  whether  she  is  in  control”  (p.  61, 
author’s  emphasis). 

•  The  loss  of  self-consciousness.  Because  attention  is  so  focused,  because  the  activity  is  so 
engrossing,  “there  is  not  enough  attention  left  over  to  allow  a  person  to  consider  either  the  past 
or  the  future,  or  any  other  temporarily  irrelevant  stimuli”  (p.  62).  When  enjoying  an  activity, 
like  a  game,  one  of  the  items  that  disappears  from  awareness  is  our  thinking  about  our  own  self. 
Loss  of  self-consciousness  “does  not  involve  a  loss  of  self,  and  certainly  not  a  loss  of 
consciousness,  but  rather,  only  a  loss  of  consciousness  of  the  self’  (p.  64,  author’s  emphasis). 

•  The  transformation  of  time.  In  an  enjoyable  activity,  “time  no  longer  seems  to  pass  the 
way  it  ordinarily  does”  (p.  66).  One  second  can  seem  to  stretch  out  for  minutes  or  conversely 
an  hour  can  seem  like  only  a  few  minutes. 

•  The  Autelic  ( intrinsically  rewarding)  experience.  An  enjoyable  activity  is  intrinsically 
rewarding.  The  word  autelic  derives  from  two  Greek  words,  auto  meaning  self  and  telos 
meaning  goal.  “It  refers  to  a  self-contained  activity,  one  that  is  done  not  with  the  expectation 
of  some  future  benefit,  but  simply  because  the  doing  itself  is  the  reward”  (p.  67). 


WORKING  DEFINITION  OF  A  GAME 

Based  on  the  above  definitions  and  for  the  purposes  of  this  paper,  the  following  working 
definition  of  game  will  be  used. 

A  game  is  an  artificially  constructed,  competitive  activity  with  a  specific  goal,  a 
set  of  rules  and  constraints  that  is  located  in  a  specific  context. 

A  game  is  not  reality.  It  is  a  constructed  activity  that  resembles  portions  of  reality.  It  provides  a 
competitive  environment  for  a  player  by  challenging  him  or  her  to  reach  a  goal.  The  purpose  of 
the  game  (e.g.,  enjoyment,  information,  instruction,  etc.)  helps  define  the  goals,  rules,  and 
context  of  the  game. 
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INSTRUCTIONAL  USES  OF  GAMES 

Most  definitions  of  games,  including  the  working  definition  above  do  not  include  any 
reference  to  instruction.  Most  games  are  played  because  they  provide  enjoyment,  not  because 
the  player  wants  to  learn  something.  However,  instruction  is  a  specific  type  of  interaction.  It  is 
an  interactive  dialogue  between  the  learner  and  the  instructional  material  (Jacobs  &  Dempsey, 
1993).  The  control  of  the  learning  experience  is  an  essential  feature  of  instruction.  Without  this 
control,  we  cannot  be  sure  that  the  student  learned  what  is  required  from  a  given  instructional 
product.  Instruction,  as  a  minimum,  must  include  the  following  four  elements.  First,  instruction 
must  be  designed  to  support  specific  instructional  objectives,  which  are  determined  by  job 
requirements.  Second,  instruction  must  include  the  opportunity  for  a  learner  to  interact  with  the 
instructional  content  in  a  meaningful  way.  Third,  the  student’s  performance  must  be  assessed  to 
determine  if  he  or  she  has  learned  what  was  intended.  Finally,  the  results  of  the  assessment  must 
be  presented  to  the  student  in  a  relevant  and  timely  manner  to  either  reinforce  correct  actions  or 
to  provide  remediation  for  incorrect  actions.  If  these  four  elements  are  not  present,  we  are  not 
dealing  with  instruction. 

Games  can  be  used  as  instructional  activities  or  some  of  the  aspects  of  games  can  be 
incorporated  into  other  instructional  activities  to  make  them  more  enjoyable  for  the  learner 
(Garris,  Ahlers,  &  Driskell,  2002).  For  example,  Parker  and  Lepper  (1992)  found  that  learners 
preferred  educational  programs  that  included  fantasy  elements.  They  also  found  that  learners 
showed  greater  learning  and  retention  in  the  fantasy  conditions  when  compared  to  activities 
without  fantasy  elements. 

Hays  and  Singer  (1989)  discussed  several  ways  that  games  could  be  used  for  instruction. 

They  observed  that  potentially,  games  can: 

•  Be  used  to  assess  entry  level  performance 

•  Measure  criterion  performance 

•  Aid  in  formative  and  summative  evaluations  of  instructional  approaches  and  programs 

•  Provide  instructional  information  on  specific  knowledge  and  skills 

•  Help  change  attitudes 

•  Serve  as  advance  organizers  prior  to  other  forms  of  instruction 

•  Replace  alternate  forms  of  instruction  to  transmit  facts,  teach  skills,  and  provide  insights 

•  Serve  as  a  means  for  drill  and  practice 

•  Help  integrate  and  maintain  skills 

•  Illustrate  the  dynamics  or  abstract  principles  of  a  task 

Based  on  their  review  of  several  studies  of  motivating  computer  games,  Malone  (1981)  and 
Malone  and  Lepper  (1987)  provide  a  framework  for  designing  intrinsically  motivating 
instructional  environments.  Many  of  these  design  heuristics  can  be  used  to  help  design 
motivational  instructional  activities  other  than  games.  However,  they  are  most  relevant  for  the 
design  of  instructional  games.  Table  1  provides  brief  descriptions  of  these  motivational 
heuristics.  The  reader  should  consult  the  original  articles  for  additional  details. 
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Table  1: 

Design  Heuristics  for  Motivating  Instructional  Environments 


INDIVIDUAL 

MOTIVATIONS 

Challenge 

Goals:  (a)  clear,  fixed  goals;  or  (b)  ability  for  learners  to 
generate  goals  for  themselves 

Uncertain  Outcomes:  fa)  variable  difficulty;  fb)  multiple 
levels  of  goals;  (c)  hidden  information,  selectively 
revealed;  and  fd)  randomness 

Performance  Feedback:  freauent,  clear,  constructive,  and 
encouraging 

Self-Esteem:  fa)  gradually  increasing  difficulty  levels  to 
promote  feelings  of  competence;  fb)  goals  that  are 
meaningful  to  the  learner 

Curiosity 

Sensorv  Curiositv:  mav  be  promoted  using  variable  audio 
&  visual  effects 

Cognitive  Curiosity:  mav  be  promoted  bv:  fa)  using 
surprise,  paradoxes,  incompleteness;  and  (b)  using 
activities  that  contain  topics  in  which  the  learner  is  already 
interested 

Control 

Contingencv:  learning  environment  should  be  responsive 
to  learner  actions 

Choice:  activities  should  provide  learner  with  choice  over 
various  aspects  of  the  learning  environment  (e.g.,  narration 
or  full  text) 

Power:  activity  should  allow  learner  to  produce  powerful 
effects 

Fantasy 

Emotional  aspects:  appeal  to  the  emotional  needs  of 
learners 

Cognitive  Aspects:  use  appropriate  metaphors  or  analogies 
for  the  material  to  be  learned 

Endogeneity:  fantasies  should  have  an  integral 
(endogenous)  relationship  to  the  material  to  be  learned 

INTERPERSONAL 

MOTIVATIONS 

Cooperation:  design  some  activities  to  promote  cooperation  among  learners 

Competition:  design  some  activities  to  require  learners  to  compete  with  one 
another  (e.g.,  actions  affect  each  other) 

Recognition:  learners’  efforts  should  receive  social  recognition  so  thev  are 
appreciated  by  others 

As  should  be  apparent  from  the  above  discussions,  it  is  difficult  to  distinguish  games  from 
other  instructional  activities  that  incorporate  game  elements.  Nevertheless,  there  have  been 
many  efforts  that  have  tried  to  evaluate  the  effectiveness  of  instructional  games.  The  next 
sections  summarize  some  of  the  research  on  the  instructional  effectiveness  of  games. 


18 


Technical  Report  2005-004 


RESEARCH  ON  INSTRUCTIONAL  GAMES 

There  have  been  many  articles  written  and  published  on  the  use  of  instructional  games.  Most 
of  this  literature  is  based  on  the  writers’  opinions  about  the  potential  of  instructional  games  (e.g., 
Driskell  &  Dwyer,  1984;  Rieber,  Smith,  &  Noah,  1998)  and  the  questions  that  must  be  answered 
about  how  games  will  be  developed  to  make  them  instructionally  sound  (e.g.,  Oblinger,  2004). 
Far  fewer  articles  have  documented  the  empirical  data  on  the  effectiveness  of  instructional 
games.  The  following  sections  provide  summaries  of  the  results  of  some  of  these  empirical 
research  efforts.  The  first  section  summarizes  the  results  of  review  articles,  which  attempted  to 
draw  general  conclusions  about  the  effectiveness  of  instructional  games  from  collections  of 
research  efforts.  Other  review  articles  are  not  summarized,  but  were  used  to  locate  original 
articles  on  research  efforts  (e.g.,  Dempsey,  Rasmussen,  &  Lucassen,  1996;  Hogle,  1996; 
Leemkuil,  do  Jong,  &  Ootes,  2000).  Subsequent  sections  provide  summaries  of  individual 
research  efforts  that  have  evaluated  specific  instructional  games  for  different  types  of  learners 
(e.g.,  school  children,  college  students,  or  military  trainees)  and  for  different  types  of  learning 
tasks  (e.g.,  factual  information,  technical  skills,  etc.).  These  summaries  focus  on  the  effects  of 
each  game  on  learner  performance  and  also,  where  possible,  describe  the  characteristics  of  the 
games  used  in  the  studies.  Where  the  information  is  available,  the  summaries  include  how  the 
game  was  used  in  a  given  instructional  program. 

REVIEW  ARTICLES  ON  THE  EFFECTIVENESS  OF  INSTRUCTIONAL  GAMES 

Simulation  games  have  been  used  extensively  in  sociology  and  social  science  courses.  Pierfy 
(1977)  concluded  from  a  review  of  22  research  studies  that  “in  terms  of  fostering  student 
learning,  simulation  games  are  no  more  effective  than  conventional  classroom  instruction”  (p. 
266).  However,  he  did  find  that  some  research  indicated  that  the  use  of  games  might  improve 
the  retention  of  learned  information.  Dom  (1989)  also  reviewed  the  use  of  simulation  games  in 
this  area.  He  maintained  that  simulation  games  are  based  on  the  model  of  experiential  learning. 
In  this  model,  learners  first  act  in  a  particular  instance  of  application.  In  the  second  stage,  they 
attempt  to  understand  the  effects  of  their  behaviors  and  decisions  in  the  particular  instance. 
Thirdly,  they  seek  to  understand  the  general  principles  under  which  that  instance  falls.  Finally, 
they  apply  the  principles  to  new  circumstances  so  that  the  learning  is  useful  to  their  future 
behavior. 

Dom  (1989)  found  mixed  results  from  his  review.  He  observed  that  there  are  consistent 
results  that  show  that  games  generate  interest  and  motivation  in  learners.  This  does  not 
automatically  produce  greater  interest  in  the  subject  matter  or  in  learning  in  general.  He  cited 
some  research  that  showed  that  simulation  games  did  not  increase  learners’  interest  in 
comparison  to  other  active  learning  techniques  like  case  studies.  However,  in  general,  he  found 
that  games  appear  to  be  as  effective  as  other  techniques  for  teaching  factual  information, 
principles,  and  concepts. 

Dom  observed  that  the  results  of  research  that  evaluated  the  ability  of  simulation  games  to 
affect  attitudes  are  confusing  and  contradictory.  Although  games  appear  to  change  attitudes,  this 
does  not  occur  every  time  a  game  is  played.  Furthermore,  some  games  have  been  shown  to 
increase  positive  attitudes  and  others  resulted  in  an  increase  in  negative  attitudes. 
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To  meet  instructional  goals,  Dorn  maintained  that  simulation  games  must  be  designed  and 
used  properly.  Instructional  games  should: 

•  Be  selected  to  accomplish  particular  goals  and  should  be  supplemented  with  discussions, 
lectures,  and  other  instructional  methods. 

•  Not  be  inserted  into  a  course  as  random  events.  Rather  they  should  be  designed  to  meet 
specific  instructional  purposes. 

•  Include  debriefing  after  the  game.  Debriefing  is  crucial  and  should  be  more  than  a  simple 
recounting  of  the  game.  It  should  be  a  structured,  guided,  activity  that  brings  meaning  to  the 
experience  and  fosters  learning  from  that  meaning. 

In  1981,  after  reviewing  books,  articles,  newsletters,  and  advertisements  about  simulation 
games,  Cathy  Stein  Greenblat  listed  the  claims  made  in  support  of  their  use.  These  claims  were 
divided  into  6  categories:  (1)  motivation  and  interest,  (2)  cognitive  learning,  (3)  changes  in  later 
course  work,  (4)  affective  learning  concerning  the  subject  matter,  (5)  general  affective  learning, 
and  (6)  changes  in  the  classroom  structure  and  relations.  She  then  summarized  the  available  data 
in  support  of  these  claims. 

The  strongest  empirical  support  was  for  the  claims  that  games  improve  learner  motivation  and 
interest  (category  1).  However,  Greenblat  observed  that  most  of  the  discussions  concerning  the 
heightened  motivation  provided  by  games  were  supported  by  anecdotal  evidence.  There  were 
some  data  to  support  the  claims  in  category  2  (cognitive  learning),  but  these  data  were  weak. 

She  found  no  evidence  to  support  the  claims  in  category  3  (changes  in  later  course  work)  and 
little  data  on  the  claims  in  category  6  (changes  in  the  classroom).  Some  evidence  indicated  that 
learners’  attitudes  about  the  subject  matter  (category  4)  could  be  changed  through  the  use  of 
games.  However,  sometimes  these  attitudes  changed  in  the  opposite  direction  to  that  which  was 
desired  by  the  game  designers.  There  was  even  less  evidence  in  support  of  general  affective 
changes  (category  5).  Her  conclusion  was  that  “there  is,  at  the  moment,  little  hard  data  to  show 
that  such  participation  leads  to  greater  interest  in  the  subject  matter,  the  course,  or  learning  in 
general”  (Greenblat,  1981,  p.  149). 

Bredemeir  and  Greenblat  (1981)  attempted  to  synthesize  the  findings  on  the  educational 
effectiveness  of  simulation  games  available  at  that  time.  They  found  little  hard  evidence  on  the 
effectiveness  of  these  games  except  for  some  support  that  games  result  in  increased  retention  of 
material.  However,  like  many  other  reviewers,  they  concluded  that  simulation  games  “are  at 
least  as  effective  as  other  methods  in  facilitating  subject  matter  learning”  (p.  165).  However, 
their  overall  conclusions  were  that  “we  do  not  yet  have  (1)  a  theoretically  based  taxonomy  of 
games  with  (2)  clear  theories  about  (a)  what  aspects  of  them  are  expected  to  have  (b)  what  sorts 
of  distinct  effects  (c)  on  what  sorts  of  students  (d)  for  what  reasons”  (p.  169). 

VanSickle  (1986)  conducted  a  quantitative  analysis  of  simulation  gaming  compared  to  other 
instructional  procedures.  He  concluded  that  there  was  weak  support  for  simulation  gaming  over 
other  approaches  (primarily  classroom  lecture).  This  conclusion  is  suspect  for  several  reasons. 
First,  he  did  not  describe  the  characteristics  of  the  simulation  games  used  in  his  analysis. 
Furthermore,  six  of  the  twenty-two  studies  did  not  compare  learners  who  participated  in 
simulation  games  with  other  forms  of  instruction.  Finally,  he  applied  several  mathematical 
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transformations  to  the  data  prior  to  drawing  his  conclusions  (e.g.,  converted  means  and  standard 
deviations  to  effect  sizes,  then  averaged  these).  These  transformations  may  have  introduced 
errors  into  his  results.  Even  without  these  cautions,  his  results  only  showed  a  moderate 
advantage  of  simulation  games.  When  looking  at  the  specific  studies  in  terms  of  learner 
performance: 

•  only  5  findings  favored  simulation  gaming  and  13  findings  showed  no  differential  impact  on 
immediate  recall  of  knowledge;  and 

•  only  2  findings  favored  simulation  gaming  and  1 5  findings  showed  no  significant  differences 
on  retention  of  knowledge. 

Thus,  from  this  review,  it  is  not  possible  to  conclude  that  simulation  gaming  is  the  preferred 
instructional  approach. 

Randel,  Morris,  Wetzel,  and  Whitehill  (1992)  reviewed  68  studies  that  compared  the 
instructional  effectiveness  of  games  to  conventional  classroom  instruction.  These  studies 
covered  a  28-year  span  of  time.  Of  the  68  studies,  38  showed  no  differences  between  games  and 
conventional  instruction;  22  favored  games  (5  additional  studies  favored  games,  but  their 
controls  were  questionable);  and  3  favored  classroom  instruction.  The  studies  were  used  to 
provide  instruction  in  social  sciences,  math,  language  arts,  logic,  physics,  and  biology.  Math 
was  the  subject  area  with  the  greatest  percentage  of  results  favoring  games.  The  review  also 
indicated  that  games  resulted  in  greater  retention  of  the  learned  information.  The  authors 
reached  several  conclusions  based  upon  their  review: 

•  “That  only  68  studies  were  reported  in  28  years  reflects  a  trend  to  use  descriptive  reports 
rather  than  empirical  studies  comparing  games  with  classroom  instruction”  (Randel,  et  al., 

1992,  p.  269). 

•  A  consistent  finding  is  that  games  are  rated  as  more  interesting  than  conventional 
instruction. 

•  Careful  consideration  should  be  given  to  the  measures  used  to  demonstrate  the  effects  of 
games.  “If  the  test  for  effectiveness  does  not  match  what  the  game  is  teaching,  negative  results 
will  occur”  (p.  271). 

•  The  experimental  designs  used  to  evaluate  games  need  to  be  more  rigorous.  Reliability 
and  validity  are  often  not  reported.  Random  sampling  is  often  not  used.  Experimental  designs 
need  to  reduce  confounding  variables  such  as,  Hawthorne  effect,  teacher  bias,  selection  effects, 
and  time  differences  for  treatments. 

Dempsey,  Lucassen,  Gilley,  and  Rasmussen  (1993-1994)  conducted  a  review  of  the  gaming 
literature.  They  collected  51  journal  articles  based  on  the  results  of  electronic  searches  (e.g., 
ERIC,  PSYCHLIT).  Most  of  the  articles  were  discussions  (n  =  28).  A  smaller  number 
summarized  research  efforts  (n  =  16).  The  remainder  were  theoretical  discussions  and  reviews 
(n=4)  or  descriptions  of  how  a  given  game  was  designed  or  developed  (n=3).  In  terms  of  the 
types  of  games  described,  simulation  games  accounted  for  the  largest  number  of  articles  (n  =  30). 
Puzzle  and  adventure  games  were  described  in  only  3  articles.  They  also  tallied  the  types  of 
learning  outcomes  and  the  functions  of  the  games  described  in  the  articles.  Table  2  shows  the 
frequency  of  articles  for  each  type  of  game  function  reported.  Table  3  shows  the  frequency  of 
articles  by  type  of  the  learning  outcome  sought  through  using  the  game. 
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Table  2: 

Percentage  of  Articles  by  Game  Function  from 
Dempsey,  et  al.  (1993-1994) 


Function  of  Game 

Percentage  of  Articles 

Leam  New  Skills 

23% 

Practice  Existing  Skills 

21% 

Not  Able  to  Determine 

14% 

Change  Attitudes 

11% 

Other 

11% 

Drill  Existing  Skills 

9% 

Tutor 

6% 

Promote  Self-Esteem 

3% 

Amuse 

2% 

Table  3: 

Percentage  of  Articles  by  Learning  Outcomes  from 
Dempsey,  et  al.  (1993-1994) 


Learning  Outcomes 

Percentage  of  Articles 

Problem  Solving 

23% 

Not  Able  to  Determine 

21% 

Attitude 

13% 

Verbal  Information 

10% 

Cognitive  Strategy 

9% 

Concrete  Concept 

9% 

Rules 

7% 

Other 

4% 

Defined  Concept 

3% 

A  few  interesting  observations  can  be  made  from  looking  at  these  frequency  data.  Table  2 
shows  that  most  games  are  used  to  teach  new  skills  or  to  practice  existing  skills.  The  next 
highest  percentage  of  game  functions  is  “not  able  to  determine.”  This  is  a  problem  with  the  way 
gaming  research  is  presented.  In  the  opinion  of  this  author,  if  one  wishes  to  publish  the  results  of 
research,  he  or  she  should  ensure  that  enough  information  is  provided  so  the  reader  can  fully 
understand  the  intent  and  results  of  the  research.  The  function  of  the  instructional  game  is 
certainly  an  important  item  of  information  if  we  wish  to  base  future  decisions  on  the  results  of 
the  research. 

In  terms  of  the  learning  outcome  of  the  games  (Table  3)  they  found  that  the  largest  number 
sought  to  help  learners  in  problem  solving  (23%),  followed  by  attitudes  (13%)  and  verbal 
information  (10%).  Again  an  unfortunate  result  is  that  like  game  functions,  a  similar 
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informational  gap  was  found  in  the  literature  on  learning  outcomes.  The  second  most  frequent 
percentage  that  Dempsey,  et  al.  (1993-1994)  found  when  looking  for  learning  outcomes  is  “not 
able  to  determine”  (21%).  If  future  instructional  game  designers  and  developers  are  to  benefit 
from  previous  research,  the  research  must  be  documented  in  a  manner  that  will  provide  all  the 
necessary  information. 

Lee  (1999)  conducted  a  meta-analysis  on  instructional  simulations  (his  term).  Nineteen 
studies  met  the  criteria  for  inclusion  in  the  analysis.  Three  modes  of  simulations  were  identified: 

•  presentations:  simulations  used  to  supplement  expository  instruction 

•  practice:  used  to  provide  learners  with  a  large  number  of  examples  and  guidance  for 

interaction  with  the  material  to  be  learned 

•  hybrids:  combinations  of  the  other  two  modes 

Lee  found  many  methodological  problems  and  confounding  variables  that  make  it  difficult  to 
draw  firm  conclusions  about  the  effectiveness  of  the  instructional  simulations.  Nevertheless, 
several  initial  conclusions  can  be  regarded  as  first  steps  toward  a  better  understanding  of 
instructional  games  in  the  future.  First,  when  used  for  presentations,  hybrid  simulations 
appeared  to  be  more  effective  than  pure  simulations.  Second,  hybrid  simulations  seem  to  be 
equally  effective  for  both  presentation  and  practice.  Third,  learners  seem  to  perform  better  when 
provided  with  specific  guidance  on  how  to  use  the  simulation. 

Egenfeldt-Nielsen  (2003)  conducted  a  review  of  the  educational  usage  of  games  by  surveying 
both  single  studies  and  other  review  articles.  The  review  focused  on  non-electronic  games,  but 
did  not  distinguish  between  simulations  and  games.  He  concluded  that  there  is  “currently  no 
evidence  for  a  better  or  worse  learning  outcome,  when  games  are  used”  (p.  1). 

Maria  Klawe,  a  senior  game  developer  stated  in  a  public  lecture  on  “The  Effective  Design 
and  Use  of  Educational  Computer  Games”  that  the  “most  common  problem  with  educational 
software  is  that  students  don’t  pay  attention  to  or  learn  the  way  that  designers  intended”  (quoted 
in  Jenson  &  de  Castell,  2002,  p.  6  of  15).  Caftori  (1994)  supported  this  view  by  providing 
several  examples  of  how  educational  software  games  do  not  always  play  an  educational  role,  at 
least  not  the  role  they  were  intended  to  play.  One  history  simulation  game  {The  Oregon  Trail 
Game )  was  intended  to  introduce  children  to  the  life  of  covered-wagon  travelers  on  their  way  to 
Oregon  in  1848.  It  contained  a  number  of  problem-solving  situations  like  crossing  a  river, 
managing  food,  or  dealing  with  disease  outbreaks.  It  also  provided  opportunities  for  the  children 
to  shoot  at  dangerous  animals  and  other  targets.  Unfortunately,  the  children  concentrated  on 
reaching  the  end  of  the  trail  as  fast  as  possible  and  shooting  animals  for  the  sake  of  shooting. 

The  critical  information  about  the  type  of  terrain  and  the  animals  associated  with  it  were  not 
noticed  and/or  learned  by  the  children.  Similar  problems  were  encountered  with  other  games. 
Even  though  the  children  interacted  with  the  games,  “they  were  able  to  do  it  in  such  a  way  that  at 
least  some  (if  not  all)  of  the  specified  educational  objectives  have  been  missed”  (Caftori  & 
Paprzycki,  1997,  p.  1  of  8).  This  is  a  conclusion  that  we  will  encounter  throughout  our  review  of 
the  literature:  an  instructional  game  will  only  be  effective  if  it  is  designed  to  meet  specific 
instructional  objectives  and  used  as  it  was  intended. 
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SUMMARIES  OF  RESEARCH  ON  SPECIFIC  INSTRUCTIONAL  GAMES 

The  following  sections  contain  summaries  of  research  on  the  instructional  effects  of  specific 
games.  The  organization  of  the  summaries  is  based  on  the  type  of  learner  and/or  the  type  of  task 
that  was  targeted  by  the  game.  The  first  section  summarizes  the  effectiveness  of  games  used  to 
teach  school  children  in  pre-kindergarten  through  the  12th  grade.  This  is  followed  by  summaries 
of  instructional  games  used  with  college  students.  Games  used  to  provide  instruction  in  the 
workplace  are  summarized  in  the  final  section. 

Games  Used  in  Schools  (Pre-K-12) 

Many  different  games  have  been  used  to  teach  a  variety  of  topics  to  school  children  in  pre¬ 
kindergarten  through  12th  grade.  The  following  sections  summarize  the  results  of  some  of  this 
research.  The  summaries  are  organized  by  topic  area. 

Games  Used  to  Teach  Math.  Koran  and  McLaughlin  (1990)  compared  the  effectiveness  of  a 
drill  and  practice  math  game  for  teaching  basic  multiplication  facts.  Twenty-eight  fifth  grade 
students  participated  in  ten  days  of  instruction.  Their  performance  was  measured  using  daily 
tests  and  a  test  at  the  end  of  the  program.  Both  the  game  and  the  drill  and  practice  conditions 
were  found  to  be  equally  effective,  but  the  students  preferred  the  game. 

Two  different  card  games  were  evaluated  in  an  elementary  school  classroom  environment  to 
help  teach  fractions,  decimals,  and  percentages  (Rowe,  2001).  The  students  played  both  games 
and  were  then  tested  on  standard  questions  about  the  math  topics.  Only  one  of  the  games, 
Percent  Rummy,  was  found  to  improve  the  learners’  performance.  The  other  game,  Find  the 
Missing  Number,  was  difficult  to  use,  did  not  stimulate  the  learners  and  did  not  result  in 
improved  performance.  These  results  indicate  that  every  game  is  not  useful  instructionally.  The 
instructional  effectiveness  of  a  game  depends  on  its  characteristics  and  how  it  is  used. 

Van  Eck  and  Dempsey  (2002)  evaluated  the  effects  of  different  ways  of  using  an  instructional 
game.  They  measured  the  math  problem-solving  skills  of  7th  and  8th  grade  students  after  they 
had  been  given  instruction  using  a  computer  simulation  game  using  competition  and  contextual 
advisement.  Some  of  the  students  experienced  the  game  in  a  “competitive”  context  and  others 
experienced  it  in  a  “non-competitive”  context.  The  competition  group  was  told  that  they  were 
competing  with  a  computer  character.  A  face  icon,  representing  the  character,  was  continuously 
shown  on  the  computer  screen.  The  non-competition  group  was  only  encouraged  to  work 
quickly  and  accurately.  Some  students  were  able  to  access  contextual  advisement  by  watching 
videos  of  computer  characters  that  provided  hints  and  encouragement.  Other  students  did  not 
have  access  to  contextual  advisement.  These  conditions  resulted  in  a  2x2  design.  The 
performance  of  these  groups  was  compared  to  an  outside  control  group  who  solved  identical 
word  problems  outside  of  a  game  context. 

The  math  problems  in  both  game  and  control  conditions  involved  determining  how  much 
paint,  wallpaper,  and  other  materials  were  necessary  to  fix  up  a  room.  The  performance  of  the 
students  was  assessed  on  a  transfer  task  requiring  them  to  apply  the  same  skills  to  determine  how 
much  paint  and  other  materials  were  required  to  fix  up  a  different  room.  The  researchers  found 
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an  interesting  interaction  between  competition  and  contextual  advisement.  The  non-competition 
group  had  higher  transfer  when  they  had  access  to  contextual  advisement,  but  the  competition 
group  had  higher  transfer  when  they  did  not  access  contextual  advisement.  Unfortunately,  these 
results  may  be  moot  because  the  control  group  performed  as  well  as  the  game  groups. 

An  Internet  game  was  used  in  conjunction  with  other  classroom  activities  to  teach 
probabilities  and  statistics  to  4-  and  5-year  old  children  (Pange,  2003).  After  a  number  of 
activities  such  as  throwing  dice  and  coins,  the  children  played  the  game  to  improve  their 
understanding  of  probabilities.  The  game  required  the  children  to  pick  one  of  three  doors  to  find 
a  hidden  car,  demonstrating  a  one-in-three  chance.  After  the  game,  the  teacher  discussed  all  of 
the  activities.  The  children  showed  improvement  in  their  understanding  of  probability.  The 
teachers  liked  using  the  game  and  reported  that  it  excited  the  students’  curiosity  and  made  the 
teaching  more  interesting.  No  determination  can  be  made  about  the  separate  instructional 
effectiveness  of  the  game. 

Laffey,  Espinosa,  Moore,  and  Lodree  (2003)  attempted  to  evaluate  the  potential  of  interactive 
computer  technology  (ICT)  for  teaching  math  skills  to  young,  low  income,  urban  children.  The 
ICT  environment  included  several  commercially  available  math  games,  such  as  Mighty  Math  and 
Millie’s  Math  House.  The  61  study  participants  were  all  Pre-K,  Kindergarten,  or  first-graders. 
The  children  were  randomly  assigned  to  treatment  (ICT)  or  control  groups  and  took  a  math 
pretest.  Some  of  the  children  were  also  identified  as  “at-risk,”  because  of  previous  behavior 
problems.  Both  groups  received  the  same  math  instruction  in  their  classroom.  The  ICT  group 
participated  in  two  20-25 -minute  ICT  sessions  per  week  over  an  eight  week  period.  Both  groups 
later  answered  posttest  questions.  Results  were  based  on  the  difference  between  pre-  and 
posttest  scores.  The  ICT  group  had  significantly  higher  gain  scores  than  the  comparison  group. 
Within  the  treatment  group,  the  not-at-risk  children  gained  more  than  the  at-risk  children.  From 
this  study,  it  is  not  possible  to  draw  conclusions  about  the  effectiveness  of  the  games  because  no 
details  are  provided.  The  results  are  also  confounded  because  the  ICT  group  received  more  math 
instruction  than  the  control  group.  The  authors  conclude  that  their  study  demonstrates  the 
“potential”  of  ICT  environments.  Like  many  other  studies  that  demonstrate  “potential,”  this 
research  does  not  provide  guidance  on  how  to  design  and  implement  instructional  games. 

A  video-game  approach  was  evaluated  for  teaching  basic  math  and  reading  comprehension 
skills  to  first-  and  second-graders  in  economically  disadvantaged  schools  in  Chile  (Rosas, 
Nussbaum,  Cumsille,  Marianov,  Correa,  Flores,  Grau,  Lagos,  Lopez,  Lopez,  Rodriguez,  & 
Salinas,  2003).  The  games  were  designed  to  play  on  a  device  that  looked  almost  identical  to  the 
popular  Gameboy  system.  The  performance  of  an  experimental  group,  who  used  the 
videogames,  was  compared  to  a  group  from  the  same  school,  who  did  not  use  the  games,  and  a 
group  from  another  school.  Although  both  groups  from  the  target  school  outperformed  the 
external  group,  there  was  no  difference  in  their  performance.  Thus,  there  is  no  support  for  the 
effectiveness  of  the  game  approach  over  traditional  instruction  on  these  basic  learning  tasks. 

Teaching  Vocabulary.  Malouf  (1987-1988)  investigated  the  motivational  effect  of  computer 
games  for  special  education  students.  He  compared  the  performance  of  sixth-,  seventh-,  and 
eighth-grade  learning-disabled  students  who  were  learning  vocabulary  skills.  His  results  showed 
no  difference  between  a  computer  program  with  game  features  and  the  same  program  without  the 
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game  features.  However,  the  game  condition  resulted  in  significantly  higher  levels  of  continuing 
motivation  than  the  non-game  condition. 

Teaching  Pedestrian  Safety.  Renaud  and  Stolovitch  (1988)  investigated  if  a  game  could  be 
used  to  help  five-year-old  children  improve  their  pedestrian  safety.  One  hundred  and  thirty-six 
5-year  olds  were  assigned  to  4  groups:  3  experimental  groups  and  1  control  group  (who  received 
no  instruction).  The  experimental  groups  included  3  variations  of  a  simulation  game.  The  first 
version  of  the  game  included  role  playing  and  interactions  among  players.  The  second  version 
of  the  game  included  behavior  modeling  and  specific  instruction  from  the  experimenter.  The 
third  game  included  all  the  elements  of  the  other  games.  The  childrens’  learning  was  measured 
using  pictures  of  a  road,  to  which  the  children  added  stickers  to  answer  a  series  of  questions.  All 
of  the  experimental  groups  outperformed  the  control  group.  The  group  that  engaged  in  role 
playing  and  player  interactions  performed  slightly  better  than  the  groups  that  experienced 
behavior  modeling. 

Using  Games  to  Study  Motivation.  Motivation  can  be  assessed  by  the  amount  of  involvement 
players  demonstrate  in  a  game.  Wishart  (1990)  examined  three  game  characteristics  (control, 
challenge,  and  complexity)  to  determine  their  effects  on  learner  involvement.  She  developed 
three  versions  of  a  role-playing  game  to  teach  children  fire  safety  knowledge.  Three  hundred 
children  (ranging  in  age  from  5  years  9  months  to  12  years  3  months  with  a  mean  age  of  8  years 
1 1  months)  participated  in  the  study.  Six  versions  of  the  game  were  developed,  each  with  a 
different  combination  of  player  control,  game  challenge,  and  game  complexity.  All  the  subjects 
took  a  pretest  on  fire  safety  knowledge  before  playing  one  of  the  versions  of  the  game.  In  the 
game,  a  player  was  presented  with  a  fire  safety  situation  and  given  a  choice  of  actions.  After 
making  the  choice,  the  player  advanced  to  another  situation.  At  the  end  of  the  game,  the  players 
answered  posttest  questions  about  fire  safety.  She  found  that  learner  involvement  was  increased 
the  most  when  the  players  had  more  control  of  the  game.  The  separate  addition  of  complexity 
and  challenge  to  control  did  not  increase  learner  involvement,  but  together  they  did. 

Furthermore,  increased  learner  involvement  was  shown  to  increase  the  improvement  of  posttest 
scores.  This  research  demonstrates  that  specific  game  designs  can  positively  affect  learning 
outcomes.  Thus,  it  is  important  to  design  instructional  games  to  increase  learner  involvement. 

Costable,  De  Angeli,  Roselli,  Lanzilotti,  and  Plantamura  (2003)  also  investigated  motivation. 
However,  they  did  not  look  at  how  a  game  motivated  learners.  Rather,  they  investigated  the 
relationship  between  learner  motivation  and  the  effectiveness  of  the  game.  They  conducted  two 
experiments  with  an  educational  software  product  called  Logiocando.  It  was  developed  to  teach 
9-10  year-olds  the  basic  concepts  of  logic.  The  product  contained  Explanation  Sections,  Logic 
Games  Sections,  and  Test  Sections.  The  games  increased  in  complexity  as  the  learners 
successfully  completed  the  simpler  exercises.  Costable,  et  al.  compared  the  performance  of 
learners  in  a  lecture  group  to  that  of  a  group  of  learners  who  used  Logiocando.  In  their  first 
experiment,  they  found  that  the  lecture  group  outperformed  the  game  group.  However,  in  their 
second  experiment,  they  specifically  motivated  the  learners  prior  to  their  interactions  with  the 
games.  They  explained  that  the  learners’  performance  would  be  carefully  monitored  by  their 
teachers  and  would  be  considered  as  part  of  their  class  work.  This  time,  the  learners  who  used 
Logiocando  performed  as  well  as  the  learners  who  received  lectures.  The  researchers  concluded 
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that  with  proper  motivation,  these  types  of  gaming  exercises  were  as  effective  as  traditional 
lecture  methods  in  helping  children  improve  their  knowledge  of  logic. 

The  Costable,  et  al.  study  introduces  an  interesting  issue  about  how  to  use  instructional 
games.  Many  people  believe  that  the  game  itself  will  motivate  learners.  However,  this  research 
indicates  that  other  sources  of  motivation  may  also  be  necessary  if  a  game  is  to  be  instructionally 
effective.  Thus,  for  games,  as  with  other  types  of  instructional  activities,  how  the  activity  is  used 
is  as  important  as  the  design  of  the  activity  itself. 

As  discussed  earlier,  Malouf  (1987-1988)  investigated  whether  playing  a  vocabulary  skill 
game  would  increase  children’s  motivation  to  continue  studying  the  topic.  He  compared  two 
groups  of  sixth-,  seventh-,  and  eighth-grade  learning  disabled  students  who  played  either  the 
game  or  used  a  computer  program  without  game  features.  He  found  that  the  game  and  nongame 
programs  produced  equal  gains  in  vocabulary  skill.  In  addition,  the  learners  who  played  the 
game  demonstrated  significantly  higher  levels  of  continuing  motivation  than  the  learners  who 
used  the  nongame  program.  This  indicates  that  instructional  games  can  have  motivational 
advantages  beyond  immediate  learning  gains. 

Games  Used  to  Teach  Geography.  Wiebe  and  Martin  (1994)  investigated  the  effectiveness  of 
a  commercial  game  for  teaching  geography  facts.  The  game,  Where  in  the  World  is  Carmen 
Sandiego,  was  compared  to  map  drawing  and  non-computer  games  (e.g.,  “Concentration”)  with 
two  groups  of  fifth-  and  sixth-grade  students.  Posttests  revealed  no  significant  differences 
between  the  two  groups  in  their  recall  of  geography  facts  or  their  attitudes  towards  studying 
geography.  The  authors  concluded  that  non-computer  games  and  activities  can  be  just  as 
beneficial  as  computer-based  games.  Unfortunately,  it  is  not  possible  to  determine  if  the  non¬ 
computer  games  or  the  map  drawing  exercises  had  separate  effects  on  learning  or  if  it  was  a 
combination  of  these  activities  that  was  as  effective  as  the  computer  game. 

A  tutoring  software  system  that  used  an  adventure  game  interface  and  adventure-type 
scenarios  was  compared  to  a  tutoring  system  that  did  not  use  these  game  aspects  for  teaching 
geography  to  fourth  graders  (Virvou,  Katsionis,  &  Manos,  2005).  Based  on  the  number  of  errors 
on  a  posttest,  it  was  found  that  the  learners  who  used  the  adventure  game  interface  improved 
their  performance  over  the  learners  who  used  the  other  system.  It  was  also  found  that  learners 
who  had  the  poorest  performance  in  geography  before  using  the  game  received  the  most  benefit. 
The  success  of  the  game  can  be  attributed  to  the  instructional  support  (e.g.,  advice,  suggestions) 
that  was  incorporated  into  the  game.  Similar  instructional  support  was  not  as  effective  in  the 
non-game  condition.  Thus,  the  combination  of  instructional  support  with  game  characteristics 
appears  to  be  an  effective  method  for  teaching  geography  principles  to  children. 

Teaching  Electronic  Circuits.  Although  not  specifically  called  a  game,  the  following  study 
provides  useful  information  that  can  apply  to  instructional  games.  A  simulation  was  used  as  a 
supplement  to  laboratory  instruction  and  exercises  to  teach  two-person  teams  of  15  year-olds 
tasks  involving  electronic  circuits  (Ronen  &  Eliahu,  2000).  The  learners  in  the  experimental 
condition  were  given  the  option  to  use  the  simulation  to  help  complete  two  exercises.  The 
learners  in  the  control  group  only  used  existing  materials  (e.g.,  workbooks).  Although  some 
indications  of  higher  achievement  were  shown  in  the  group  that  used  the  simulation,  the  authors’ 
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observations  about  learners  who  did  not  benefit  from  the  simulation  may  be  more  interesting. 
They  observed  that  the  simulation  was  not  effective  for  three  groups: 

•  Learners  who  did  not  need  additional  assistance  (about  10%  of  the  total)  and  did  not  use  the 
simulation.  Some  did  use  it  in  later  more  advanced  tasks. 

•  Learners  with  insufficient  understanding  of  the  domain  (about  10-15%),  who  only  performed 
random  trials  on  the  simulation  rather  that  actions  that  would  help  them  on  their  specific  tasks. 

•  A  few  students  would  not  use  the  computer  (about  5%)  because  they  “hated  computers.” 

Teaching  Physics.  White  (1984)  used  a  series  of  simple  force  and  motion  video  games  to 
teach  high-school  students  the  principles  of  Newtonian  physics.  The  games  all  used  a 
representation  of  a  space  ship  to  illustrate  the  effects  of  force  on  motion  in  a  frictionless 
environment.  The  games  required  the  students  to  move  the  space  ship  to  various  targets  by 
applying  different  amounts  and  directions  of  force.  The  performance  of  the  students  on  a  series 
of  posttest  force  and  motion  problems  was  compared  to  a  group  that  received  no  instruction 
(both  groups  scored  similarly  on  a  pretest).  The  students  who  played  the  games  improved  their 
answering  of  the  force  and  motion  problems  more  than  those  who  did  not  play  the  games.  This 
study  does  show  that  these  games  can  provide  effective  instruction  on  Newtonian  force  and 
motion  principles.  However,  there  was  no  comparison  to  other  instructional  approaches,  so  it  is 
not  possible  to  conclude  that  games  are  the  most  effective  way  to  teach  these  principles. 

Use  of  Games  for  Health  Care  Education.  Games  have  been  used  to  help  teach  people  about 
health  care  issues  and  the  health  care  system.  Sleet  and  Stadskley  (1977)  provided  an  annotated 
bibliography  of  66  simulation  games  used  in  health  education.  These  games  were  used  in  a 
variety  of  health  care  areas  including:  disease  management;  drug  use  and  abuse;  ecology;  family 
planning  and  human  sexuality;  nursing;  nutrition;  physical  fitness;  and  safety.  The  games 
allowed  individuals  to  experience  various  roles  and  requirements  of  both  health  care  personnel 
and  their  patients.  These  authors  provided  no  information  on  the  instructional  effectiveness  of 
the  games. 

Thomas,  et  al.  (1997)  used  an  interactive  computer  game,  Life  Challenge,  as  a  tool  to  enhance 
adolescents’  sense  of  self-efficacy  in  HIV/AIDS  prevention.  The  game  allowed  learners  to 
travel  to  different  times  and  role  play  with  imaginary  partners,  then  to  hear  their  recorded 
statements  and  decide  to  “try  again”  or  stay  with  their  original  statements.  There  was  also  a 
game  show  segment  where  the  learners  picked  the  contestant  they  felt  gave  the  best  line  in  each 
situation.  A  pretest,  posttest  comparison  of  true-false  questions  was  used  to  determine  if  there 
were  learning  gains  on  7  knowledge  items.  Learning  gains  were  found  on  3  of  the  7  items. 
Learners  also  showed  improved  self-efficacy  when  their  pre-test  and  posttest  scores  were 
compared  on  several  self-efficacy  items.  All  learners  showed  increased  self-efficacy  with  the 
strongest  gains  shown  by  those  who  began  with  the  lowest  self-efficacy  scores.  Unfortunately,  it 
is  not  possible  to  attribute  these  effects  to  the  game  alone  since  there  were  no  comparisons  to 
other  instructional  methods. 

Brown,  Lieberman,  Gemeny,  Fan,  Wilson,  and  Pasta  (1997)  assessed  the  effectiveness  of  a 
role-playing  video  game  to  teach  young  diabetic  patients  about  diabetes.  They  compared  two 
groups  of  young  diabetics  (ages  8  to  16).  One  group  played  an  interactive  adventure  video  game 
(Packy  &  Marlon),  in  which  they  played  the  role  of  a  character  that  had  diabetes.  The  other 
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group  played  a  video  game  containing  no  diabetes-related  content.  The  two  groups  played  their 
games  at  home  as  often  as  they  wished.  They  were  interviewed  and  their  parents  filled  out  a 
questionnaire  at  the  beginning  of  the  study,  after  three  months,  and  after  six  months.  After  six 
months,  the  group  that  played  the  diabetes  game  showed  higher  perceived  self-efficacy  for 
diabetes  self-management,  increased  their  communication  with  their  parents  about  diabetes,  and 
improved  their  daily  diabetes  self-management  behaviors  (e.g.,  monitoring  the  blood  glucose 
levels  and  taking  insulin  as  needed).  The  control  group  did  not  change  their  behaviors. 

Although  the  game  shows  promise,  the  study  did  not  compare  game  use  with  other  instructional 
approaches. 

Similar  conclusions  can  be  drawn  about  the  evaluation  of  a  game  to  teach  bilingual  children 
about  the  food  pyramid  and  better  eating  habits.  Serrano  and  Anderson  (2004)  found  that  fifth 
graders  had  better  knowledge  about  nutrition  and  the  food  pyramid  after  playing  a  game  when 
compared  to  a  group  that  had  no  instruction.  This  shows  that  a  game  can  be  better  than  nothing, 
but  does  not  help  us  decide  whether  using  a  game  is  the  most  effective  instructional  approach. 

Games  Used  in  College 

Teaching  Social  Science.  Szafran  and  Mandolini  (1980)  examined  the  effectiveness  of  a 
simulation  game  called  SIMSOC:  Simulated  Society  for  teaching  undergraduates  in  an 
introductory  sociology  course.  The  non-computer  game  was  intended  to  create  a  situation  where 
the  student  examined  the  processes  of  social  conflict  and  social  control  in  a  simulated  society. 
The  results  of  the  game  were  compared  to  the  performance  of  students  who  received 
conventional  classroom  instruction.  Performance  was  evaluated  in  terms  of  improvements  in  the 
students’  test  scores  and  their  ability  to  recognize  sociological  concepts  in  nonsociological 
writings  (e.g.,  excerpts  from  a  novel,  a  magazine  story,  and  a  newspaper  article).  No  statistically 
significant  differences  were  found  in  the  performance  of  the  two  groups.  They  concluded  that 
“there  continues  to  be  no  evidence  that  simulation  games  substantially  increase  cognitive 
knowledge”  (p.  334). 

Teaching  Abnormal  Psychology.  Brewster  (1996)  used  games  in  both  an  interactive 
multimedia  computer  environment  and  in  a  more  traditional  classroom  environment  to  teach 
undergraduates  the  principles  of  abnormal  psychology.  The  games  were  modifications  of 
Concentration  and  Jeopardy.  She  found  no  differences  among  the  groups  on  any  measure  of 
performance.  It  appears  that  multimedia  is  not  necessary  for  games  to  provide  an  effective 
instructional  approach.  However,  this  study  does  not  tell  us  when  to  use  the  games  rather  than 
other  instructional  methods. 

Teaching  Economics.  Fraas  (1982)  compared  a  simulation  gaming  approach  to  the  lecture- 
discussion  method  for  teaching  college-level  economics  principles.  The  game  condition 
consisted  of  seven  commercial  simulation  games  used  in  succession.  Unfortunately,  detailed 
descriptions  of  the  games  were  not  provided  in  this  publication.  He  found  that  neither  method 
was  superior  to  the  other  in  overall  effectiveness.  However,  he  also  found  that  learners  with  less 
previous  knowledge  of  economics  showed  better  improvement  with  the  simulation  game 
approach.  This  was  also  true  of  students  with  lower  SAT  scores.  The  reverse  was  also  found. 
Students  with  higher  SAT  scores  and/or  more  previous  economic  knowledge  appeared  to  learn 
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better  from  the  lecture-discussion.  This  study  indicates  that  games  may  be  differentially 
effective  for  learners  with  different  abilities  or  experiences. 

Gremmen  and  Potters  (1995)  evaluated  whether  lectures  supplemented  with  a  game  were 
more  effective  for  teaching  economics  principles  than  lectures  alone.  They  compared  the 
performance  of  two  groups  on  a  multiple-choice  economics  test.  The  group  who  received  a 
lectures  supplemented  with  the  game  outperformed  the  group  that  only  received  the  lectures. 

Reasoning  Skills.  Wood  and  Stewart  (1987)  tried  to  determine  if  use  of  a  computerized 
version  of  the  game  Mastermind  would  help  college  students  improve  their  skills  in  practical 
reasoning.  Thirty  students  were  divided  into  two  groups.  Both  groups  took  a  pre  and  posttest  on 
the  Watson-Galser  Thinking  Appraisal.  The  experimental  group  played  the  game  between  test 
administrations,  but  the  control  group  received  no  treatment.  The  experimental  group  reduced 
their  errors  on  the  posttest  relative  to  the  control  group  and  indicated  that  they  enjoyed  the 
experience.  However,  it  is  not  clear  whether  the  game  directly  affected  these  results  or  if  they 
were  changed  due  to  repetition  of  the  test  and/or  more  time  on  the  task.  If  the  game  did 
influence  the  results,  it  is  still  not  clear  what  game  characteristics  might  have  affected  the 
learners’  performance. 

Teaching  About  Learning.  Klein  and  Freitag  (1991)  compared  the  instructional  effectiveness 
of  using  a  board  game  to  the  use  of  a  worksheet  for  teaching  the  principles  of  the  information¬ 
processing  model  of  learning  to  undergraduate  education  majors.  The  game  and  worksheet  were 
designed  to  supplement  and  provide  practice  on  material  from  assigned  textbook  reading.  The 
board  game  included  a  set  of  25  game  cards  with  practice  questions  about  the  topic.  The  rules  of 
the  game  “were  developed  to  encourage  cooperation,  competition,  and  active  participation”  (p. 
304).  The  worksheet  included  the  same  25  questions  that  appeared  on  the  game  cards.  No 
performance  differences  were  found  as  a  result  of  the  two  methods  of  instruction.  However,  the 
learners  who  reported  that  they  had  read  the  textbook  assignment  performed  better  than  those 
who  did  not  complete  the  assignment. 

Games  Used  to  Change  Attitudes.  Williams  (1980)  evaluated  the  effectiveness  of  a 
simulation  game  to  change  the  attitudes  of  college  students  toward  a  negatively-presented 
historical  character.  The  students  first  read  a  fictional  history  of  a  struggle  among  four 
noblemen.  One  nobleman  was  presented  as  a  “villain.”  After  the  reading,  they  all  filled  out  an 
attitude  questionnaire  about  the  characters.  Then  they  were  separated  into  two  experimental 
groups  and  one  control  group.  The  control  group  engaged  in  a  non-related  activity  while  the  two 
experimental  groups  engaged  in  a  simulation  game  in  which  they  had  to  “play”  the  role  of  the 
negative  character.  One  experimental  group  (dissonance)  was  not  given  praise  or  reward  for 
playing  the  character.  The  other  experimental  group  (incentive)  was  given  praise  and  a 
monetary  reward  for  their  role  play.  It  was  assumed  that  cognitive  dissonance  would  increase  in 
the  group  that  had  to  play  the  negative  character  with  no  outside  incentives.  Both  of  the 
experimental  groups  changed  their  attitudes  toward  the  negative  character  to  a  significantly 
greater  degree  than  the  students  in  the  control  group.  No  differences  were  found  in  the  attitude 
change  between  the  two  experimental  groups. 
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Another  study  (Bredemeier,  Bernstein,  &  Oxman,  1982)  found  that  a  game  designed  to 
change  college  students’  attitudes  on  dogmatism  and  ethnocentrism  was  effective  when  used  in 
an  anthropology  class  compared  to  another  anthropology  class  that  did  not  use  the  game. 
However,  the  same  game,  used  in  a  different  course  (philosophy)  did  not  result  in  the  same 
degree  of  attitude  change.  These  results  indicate  the  importance  of  placing  instructional  games 
in  the  appropriate  instructional  context. 

Teaching  Empathic  Understanding.  A  role-playing  game  was  compared  to  classroom  lectures 
for  improving  the  empathic  understanding  of  first-year  university  students  (Barak,  Engle,  Katzir, 
&  Fisher,  1987).  The  game  was  designed  to  improve  the  communication  skills  involved  in 
empathy  (understanding  another  person  and  communicating  this  understanding  to  him  or  her).  It 
involved  analysis,  diagnosis,  and  understanding  a  target  “client.”  Learners  in  the  game  group 
took  turns  playing  the  role  of  the  “client”  while  other  players  listened  to  the  “client”  describing 
his  or  her  situation.  The  control  group  participated  in  a  lecture  and  group  discussion  about 
listening  and  understanding  others  feelings.  The  performance  of  the  experimental  group 
significantly  improved,  but  the  performance  of  the  control  group  did  not  change.  The  authors 
concluded  that  the  game  was  an  effective  method  for  improving  empathic  skills. 

Teaching  Physics,  Electronics,  and  Engineering  Principles.  Rieber  and  Noah  (1997)  used  a 
game  to  teach  university  students  the  relationships  between  acceleration  and  velocity.  When 
compared  to  participants  who  were  taught  these  topics  without  a  gaming  context,  it  was  found 
that  the  game  actually  interfered  with  the  participants’  learning  of  these  principles.  The  authors 
examined  several  patterns  of  interaction  with  the  game  that  may  have  led  to  this  outcome.  The 
participants  who  were  given  assistance  by  an  outside  agent  were  better  able  to  consider  the 
relevancy  of  the  game  for  learning  the  science  principles.  “When  left  on  their  own,  participants 
had  much  difficulty  in  focusing  their  attention  on  how  to  manipulate  the  task  in  order  to  learn 
more  about  the  content”  (p.  8).  Without  assistance,  “participants  focused  far  too  much  on  the 
competitive  nature  of  the  game  and,  as  a  result  their  ability  to  monitor  their  own  comprehension 
was  inhibited”  (p.  8).  This  study,  although  exploratory  in  nature,  illustrates  the  important  issue 
of  the  instructional  support  that  surrounds  a  game  and  how  the  game  is  used  to  support 
instruction.  Without  instructional  support,  a  game  may  be  not  only  ineffective,  but  it  may  be 
detrimental  to  learning. 

Miller,  et  al.  (1999)  examined  three  versions  of  an  instructional  simulation  game  (Electronic 
Field  Hockey)  to  determine  which  was  most  effective  in  helping  undergraduate  students  develop 
a  better  understanding  of  the  physics  of  electrical  interactions.  The  simulation  was  intended  to 
allow  learners  to  observe  how  the  trajectory  of  a  “puck”  with  an  electrical  charge  was  influenced 
by  obstacles  with  various  electrical  charges.  The  three  versions  of  the  simulation  were: 

•  A  no-goal  condition,  which  included  no  obstacles,  no  net,  and  no  specific  task. 

•  A  standard-goal  condition,  in  which  the  learners’  task  was  to  try  to  position  the  obstacles  so 
that  their  charges  would  cause  the  “puck”  to  avoid  them  and  go  into  the  hockey  net. 

•  A  specific  path  condition,  which  showed  the  learners  an  ideal  trajectory  which  they 
attempted  to  duplicate.  It  also  showed  the  difference  between  the  ideal  trajectory  and  their 
actual  trajectory  that  was  obtained  by  their  placement  of  the  obstacles. 
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Both  the  no-goal  and  specific-path  conditions  produced  higher  learner  scores  than  the 
standard-goal  condition.  The  authors  concluded  that  these  two  conditions  required  learners  to 
engage  in  broader  exploration  and  to  be  more  selective  in  interpreting  the  evidence  than  the 
standard-goal  condition.  The  study  highlights  the  importance  of  carefully  selecting  and 
analyzing  the  intended  goals  of  an  instructional  game  before  it  is  implemented. 

Crown  (2001)  conducted  a  study  to  determine  the  effect  of  web-based  games  on  the 
visualization  skills  of  engineering  graphics  students.  However,  it  is  difficult  to  determine  the 
effect  of  his  instructional  games  because  the  game  pages  were  only  part  of  an  instructional  CD. 
Although  learners  who  used  the  instructional  CD  showed  positive  performance  on  exams,  it  is 
not  possible  to  separate  the  effect  of  the  game  pages  from  the  effect  of  other  instructional 
material  presented  on  the  CD. 

Teaching  Principles  of  Underwater  Sound  Propagation.  Shrestha  (1990)  compared  a  simple 
game  to  more  traditional  lecture-type  instruction  on  the  principles  of  underwater  sound 
propagation.  She  found  no  learning  differences  among  two  groups  of  college  students. 

However,  she  did  find  that  the  learners  in  the  game  condition  spent  more  time  interacting  with 
the  game.  She  concluded  that  games  may  have  motivational  benefits  over  the  traditional 
instruction. 

Games  to  Teach  Health  Care  Decisions.  Westbrook  and  Braithwaite  (2001)  evaluated  a  web- 
based  educational  game  to  help  college  undergraduates  make  better  health  care  decisions.  The 
game  presented  four  families  in  a  series  of  health  care  events.  It  then  generated  problems  that 
required  the  learners  to  seek  out  information  to  assist  the  families  in  making  decisions  like  which 
type  of  care  to  seek  and  how  much  it  would  cost.  The  participants  were  55  students  from 
Australia  and  South-East  Asia.  They  worked  in  competing  teams  of  six  persons.  Each  team 
member  contributed  to  a  common  discussion  log  over  a  four  week  period.  The  evaluation 
consisted  of  a  pre-  and  post-questionnaire  that  assessed:  (1)  the  participants’  interest  in  and 
knowledge  of  the  health  care  system,  (2)  their  experience  with  computers,  (3)  their  views  on 
team  work,  and  (4)  their  demographics.  Only  two  questions  assessed  the  learners’  factual 
knowledge. 

Although  a  higher  percentage  of  the  participants  correctly  answered  the  factual  questions  on 
the  posttest,  these  results  were  highly  confounded.  Prior  to  the  game,  the  participants  had 
already  received  9  weeks  of  lectures  which  included  the  topics  covered  in  the  game.  The  authors 
concluded  that  the  improvement  on  the  factual  questions  indicated  the  effectiveness  of  the  game. 
However,  other  events  during  the  13  week  period  (9  weeks  of  lectures  and  4  weeks  of  game 
participation)  could  have  influenced  their  answers. 

The  Westbrook  and  Braithwaite  (2001)  study  illustrates  many  of  the  problems  that  make  it 
difficult  to  reach  conclusions  about  either  the  effectiveness  of  a  specific  instructional  game  or  the 
effectiveness  of  games  in  general.  Some  of  these  problems  include: 

•  The  evaluators’  interest  in  “proving”  the  effectiveness  of  their  game  (i.e.,  they  developed 
the  game  and  wanted  it  to  work). 

•  No  control  group  who  experienced  an  alternate  form  of  instruction  to  compare  to  the 
game. 
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•  No  clear  and  detailed  description  of  the  game  itself  (e.g.,  what  characteristics  made  it  a 
“game”). 

•  Lack  of  control  of  possible  confounding  variables  (e.g.,  events  prior  to  and  during  the 
evaluation). 

Teaching  Marketing.  Business,  and  Management  Principles.  Fritzsche  (1981)  compared  the 
use  of  a  marketing  simulation  game,  Marketing  in  Action,  as  either  the  central  delivery  vehicle 
for  instructional  information  or  as  a  supplement  to  the  standard  lecture  on  the  same  material.  No 
differences  were  found  between  the  two  conditions  on  college  students’  midterm  and  final 
examination  grades.  Thus,  Fritzsche  concluded  that  the  game  alone  was  as  effective  as  the 
lecture  with  the  game  as  a  supplement.  The  equivalence  in  effectiveness  may  be  because  the 
game  (Day,  1962)  included  detailed  instructions  on  how  and  why  to  fill  out  the  simulated 
management  decision  fonns.  These  may  have  had  as  much  effect  on  later  performance  as 
playing  the  game  itself. 

Wellington  and  Faria  (1996)  studied  the  effect  of  team  cohesion,  player  attitudes,  and 
performance  expectations  on  the  instructional  effectiveness  of  a  simulation  game.  Although  they 
do  not  specifically  describe  the  simulation  games,  it  appears  to  be  a  marketing  simulation 
specifically  developed  for  an  introductory  marketing  course.  The  game  was  played  by  389 
students  from  two  sections  of  a  Principles  of  Marketing  course.  They  were  divided  into  108 
teams  of  three  or  four  players.  The  teams  competed  with  one  another  and  each  team  had  to  make 
six  decisions  during  the  competition.  The  teams  that  were  more  cohesive  at  the  start  of  the 
competitions  outperformed  the  teams  that  were  less  cohesive.  The  game  did  not  appear  to 
change  the  cohesiveness  of  the  teams  over  the  course  of  the  competition.  Furthermore,  learner 
attitudes  were  not  found  to  be  statistically  related  to  final  game  performance  although  learner 
expectations  were  positively  related  to  performance.  Unfortunately,  this  research  tells  us  very 
little  about  the  specific  characteristics  of  the  game.  It  also  does  not  compare  the  use  of  the  game 
with  other  instructional  approaches. 

Rowland  and  Gardner  (1973)  did  not  find  positive  support  for  the  instructional  value  of  a 
business  and  management  game.  In  their  study  of  200  college  students  who  played  the  Marksim 
business  game  as  seven-person  teams,  there  was  no  positive  relationship  between  playing  the 
game  and  grades  in  a  marketing  course.  They  concluded  that  business  games  were  not  the 
educational  panacea  envisaged  by  many  advocates. 

Games  Used  in  the  Workplace 

Teaching  Attention  Allocation  Skills.  Gopher,  Weil,  and  Barebet  (1994)  found  positive 
learning  effects  from  a  video  game  called  Space  Fortress.  They  found  that  attention  allocation 
skills  could  be  trained  using  the  game.  These  attention  allocation  skills  were  found  to  transfer  to 
piloting  skills  in  complex  fighter  aircraft.  However,  these  results  are  ambiguous  because  other 
training  (e.g.,  verbal  tips  and  other  part-task  training  sessions)  may  also  have  affected  the 
outcomes. 

Teaching  Periscope  Skills.  Garris,  et  al.  (2002)  described  an  initial  evaluation  of  a  game 
called  BOTTOM  GUN.  It  was  developed  to  help  Navy  personnel  learn  periscope  skills  such  as 
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distance  estimation  and  angle-on-the-bow  (i.e.,  the  angle  at  which  an  observed  ship  is  visually 
presented  to  the  periscope  observer).  The  game  was  designed  to  incorporate  game 
characteristics  such  as  curiosity,  competition  and  control,  as  well  as,  visual  and  sound  effects. 
Although  an  initial  evaluation,  their  results  indicated  that  learners  using  the  game  showed  more 
improved  performance  than  those  using  a  trainer  without  game  elements. 

Teaching  Principles  of  Chemical.  Biological,  and  Radiological  Defense.  Ricci,  Salas,  and 
Cannon-Bowers  (1996)  investigated  the  learning  effects  of  a  very  simple  game.  They  compared 
three  groups  of  Navy  trainees  (mean  age  =  20  yrs.)  who  were  taught  the  basic  principles  of 
chemical,  biological,  and  radiological  defense.  One  group  (text  group)  studied  a  printed  text  (a 
63 -page  pocket  handbook).  The  second  group  (test  group)  was  given  a  printed  copy  of  88 
multiple-choice  questions  with  answers.  The  third  group  (game  group)  interacted  with  a 
computer  program  that  presented  the  88  questions  in  random  fashion  in  boxes  that  resembled  a 
slot  machine.  This  group  was  given  3  minutes  to  answer  each  question  and  received  points  for 
correct  answers.  Both  the  test  and  game  groups  performed  better  than  the  text  group  on  a 
posttest.  However,  no  performance  differences  were  shown  between  the  test  and  game  groups 
although  the  game  group  scored  higher  on  a  retention  test  given  four  weeks  later.  The  results  of 
this  study  are  questionable  because  the  game  condition  was  basically  a  way  to  randomize  the 
presentation  of  the  questions  and  may  not  have  been  a  true  game. 

Teaching  Technical  Skills.  Oxford,  Harman,  and  Holland  (1987)  reported  the  initial 
evaluations  of  two  hand-held  computerized  game-based  training  aids  for  Army  trainees.  The 
two  training  aids,  called  Tutor  and  CHIP,  used  simple  games  (e.g.,  matching  pictures  to  verbal  or 
visual  stimuli  to  send  a  projectile  against  an  “enemy”  target)  to  teach  technical  terminology  and 
basic  technical  skills.  Learners  who  used  the  aids  completed  more  instructional  units  than  a 
group  who  used  a  workbook  containing  the  same  content.  The  training  aid  group  also 
outperformed  the  workbook  group.  Although  these  data  were  very  tentative  and  the  games  were 
simplistic,  they  did  show  the  instructional  potential  of  gaming  activities. 

Whitehill  and  McDonald  (1993)  compared  the  effects  of  the  presentation  of  circuit  repair 
problems  in  a  drill  format  to  presentation  of  the  same  problems  in  a  game  context.  The  drill 
context  simply  presented  the  circuit  problems  one  at  a  time.  However,  the  game  context  required 
learners  to  simulate  the  role  of  an  electrician  repairing  the  circuits.  They  used  a  computer  to 
move  a  cursor  around  a  Navy  ship’s  floor  plan  to  locate  and  solve  the  problems.  Video  game 
sound  effects  accompanied  the  movement  of  the  cursor  through  the  ship.  The  payoff  for  solving 
the  problems  was  either  fixed  or  variable  based  on  problem  difficulty.  Although  they  found  no 
performance  differences  between  learners  in  either  the  drill  or  game  conditions,  they  did  find 
that  combining  the  game  with  variable  payoff  resulted  in  increased  learner  persistence. 

Parchman,  Ellis,  Christinaz,  and  Vogel  (2000)  evaluated  four  different  instructional 
conditions,  including  a  game,  to  train  basic  electricity  and  electronics  fundamentals  to  beginning 
Navy  electronic  technicians.  They  compared  conventional  classroom  instruction  (Cl)  with 
computer-based  drill  and  practice  (CBDR),  enhanced  computer-based  instruction  (ECBI)  that 
added  compelling  graphics,  animations,  and  simulations  to  the  CBDR,  and  a  role-playing 
adventure  game.  The  trainees  in  the  CBDR  and  ECBI  conditions  outperformed  those  trained  in 
either  the  conventional  instruction  or  the  game.  The  CBDR  and  ECBI  conditions  were  equally 
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effective  in  teaching  symbols  and  definitions.  However,  cause-and-effect  relationships  were 
learned  better  by  trainees  in  the  ECBI  condition.  This  is  probably  because  of  the  enhanced 
visualizations  provided  by  the  ECBI  program.  The  ECBI-trained  learners  were  also  more 
confident  than  those  in  the  other  groups  that  they  had  learned  the  required  material. 

These  researchers  concluded  that  instructional  developers  need  to  be  cautious  and  not  rush 
into  exclusively  using  games  until  additional  data  are  available  on  their  effectiveness.  Other 
computer-based  instructional  approaches  may  be  more  effective  than  games  for  certain  types  of 
instruction.  They  also  recommended  that  instructional  developers  not  overdevelop  computer- 
based  instruction.  Simple  computer-based  instruction  may  be  sufficient  when  the  objective  is  to 
teach  definitions  and  symbols.  When  more  complex  material  needs  to  be  learned  (e.g.,  cause- 
and-effect  relationships),  enhanced  computer-based  instruction  may  be  required. 

Business  and  Management  Games.  Gaming  has  been  used  to  teach  business  and  management 
skills  for  many  years.  Faria  (1989)  observed  that  business  games  are  direct  descendents  of  war 
games,  dating  back  at  least  to  the  German  Kriegspiel  of  the  mid- 19  century.  He  also  observed 
that  the  RAND  Corporation  developed  an  instructional  simulation  game  in  1955.  Called 
Monopologs,  it  was  intended  to  allow  U.  S.  Air  Force  logistics  personnel  to  make  logistics 
decisions  in  a  “risk-free”  environment.  No  data  could  be  located  on  the  effectiveness  of  the 
game. 

The  first  widely  known  business  game  was  developed  by  the  American  management 
Association  in  1956.  Called  AM  A ’s  Top  Management  Decision  Simulation,  it  provided  an 
environment  in  which  two  teams  of  players  could  represent  officers  of  firms  and  make  business 
decisions  (Cohen  &  Rhenman,  1961).  Five  teams  with  three  to  five  persons  each  produced  a 
single  product  which  they  sold  in  competition  with  other  teams.  Many  varieties  of  this  game 
were  developed  in  subsequent  years,  although  no  data  on  its  effectiveness  were  found. 

In  the  early  1960s,  a  game  was  used  to  teach  diplomats  international  relations  skills  (Benson, 
1961).  The  Simple  Diplomatic  Game  reproduced,  in  simplified  form,  features  of  the 
international  political  arena  and  allowed  learners  to  see  the  effects  of  their  actions  in  a  larger 
action-counteraction  cycle.  No  data  were  found  on  its  effectiveness. 

Faria  (1989)  conducted  surveys  of  training  managers,  businesses,  and  business  schools  on 
their  use  of  games.  Of  the  223  training  managers  who  responded,  54.7%  indicated  that  their 
companies  used  simulation  games  in  their  training  programs.  Results  also  indicated  that  at  least 
5000  firms  were  using  business  games  and  1,700  four-year  business  schools  used  games  in  their 
programs.  Wolfe  (1997)  observed  that  by  1974  hundreds  of  articles  on  business  gaming  had 
appeared  in  the  business  and  professional  press. 

Cohen  and  Rhenman  (1961)  and  Morris  (1976)  maintain  that  all  management  games  share 
several  common  features. 

•  They  allow  the  presentation  of  feedback  based  on  the  results  of  players’  actions. 

•  They  represent  the  environment  in  logical  or  mathematical  relations.  Some  of  these  relations 

are  known  to  the  players  as  “rules”  while  others  are  only  vaguely  qualified  and  become  known 

during  play. 
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•  They  allow  interaction  between  players  and  the  environment.  Sometimes  players  can  assume 
the  roles  of  managers  in  different  functional  areas  within  a  company  to  learn  how  these  areas 
differ  from  their  own. 

•  They  provide  a  simplified  view  of  reality  so  players  can  focus  on  single  issues  or  areas  as 
they  learn. 

Citing  Cohen  and  Rhenman  (1961)  and  Morris  (1976),  Hays  and  Singer  (1989,  pp.  198-200) 
mentioned  several  educational  properties  of  business  and  management  games: 

•  They  can  provide  training  on  the  interactions  of  functional  specialties  within  companies. 
The  players  can  learn  how  their  specialty  is  dependent  on  others,  and  how  to  interact  more 
effectively  to  reach  common  goals. 

•  Playing  the  game  may  sensitize  learners  to  the  fact  that  in  the  real  world  they  must  take 
particular  actions  solely  for  the  sake  of  information  gathering. 

•  Games  can  offer  the  trainee  the  opportunity  to  learn  and  refine  a  variety  of  analytic  tools. 

•  Games  may  allow  learners  to  become  aware  of  the  psychological  and  organizational 
interactions  and  interdependencies  in  business. 

•  Players  may  learn  that  most  decisions  are  made  by  teams  of  several  persons  and  that 
these  decisions  are  constrained  by  a  variety  of  variables,  such  as  time,  complexity  of  the 
decision,  and  personality  factors  of  the  players. 

•  Games  can  teach  institutional  facts  about  the  learners’  company  (e.g.,  standard  operating 
procedures). 

These  properties  and  many  of  those  mentioned  by  other  authors  also  apply  to  games  outside  of 
business  and  management. 

Thornton  and  Cleveland  (1990)  described  management  gaming  simulations  as  a  hierarchy  of 
increasing  complexity.  Their  hierarchy  included: 

•  One-on-one  interview  simulations  that  are  “short,  yet  powerful,  simulations  of  specific 
interpersonal  interactions”  (p.  192).  The  authors  list  several  studies  that  indicated  that  these 
were  effective  in  enhancing  supervisory  skills.  However  they  also  cited  contradictory 
evidence  that  showed  no  behavior  change  as  a  result  of  participating  in  this  type  of  gaming 
simulations. 

•  Leaderless  group  discussions  simulate  “numerous  ad  hoc  committee  situations  in 
organizations  in  which  problem  analysis  and  decision  making  take  place”  (p.  192).  The 
authors  cited  no  specific  data  on  the  instructional  effectiveness  of  these  discussion  simulations. 

•  In-basket  technique.  A  type  of  simulation  named  after  the  in-tray  on  a  manager’s  desk  has 
been  used  to  practice  administrative  skills.  Although  the  technique  has  been  shown  to  have 
predictive  validity  and  is  used  in  “virtually  all  assessment  centers”  (p.  193)  for  managerial 
selection,  Thornton  and  Cleveland  cited  no  data  on  their  instructional  effectiveness. 

•  Complex  decision-making  simulations  have  been  used  to  study  the  components  of  managerial 
decision  making  (Streufert,  1986).  However,  Thornton  and  Cleveland  found  no  evaluations  of 
their  instructional  effectiveness. 

•  Large-scale  behavioral  simulations  involve  “multiple  business  problems  and  opportunities, 
intensive  and  extensive  interaction  among  managers,  and  observation  by  trained  staff’  (p.  193- 
194).  Little  published  empirical  information  was  available  on  these  simulations. 
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Morris  and  Holman  (1976)  report  the  successful  application  of  business  gaming  at  a  large 
pharmaceutical  company.  A  business  management  competition  was  held  with  638  people 
divided  into  192  teams.  These  teams,  from  various  divisions,  learned  an  appreciation  of 
business,  as  a  whole,  rather  than  just  their  work  functions.  Some  of  the  lessons  learned  included: 
methods  of  increasing  profits  by  either  increasing  income  or  decreasing  expenditures;  team  work 
skills;  methods  to  get  along  with  the  competition;  and  marketing  techniques. 

Carlson  and  Hill  (1982)  examined  the  effectiveness  of  a  gaming  approach  to  reducing 
absenteeism  and  tardiness  at  a  small  manufacturing  firm.  They  found  that  the  impact  of  gaming 
was  minimal  on  these  two  areas.  However,  they  did  find  that  when  gaming  was  used  as  a 
vehicle  for  employee  communication,  it  resulted  in  statistically  significant  improvement  in 
employee  attitudes  and  cooperation. 

Wolfe  (1997)  conducted  a  review  of  business  games  used  to  teach  strategic  management 
skills.  He  reviewed  seven  studies  conducted  between  1966  and  1988  that:  1)  compared  game  use 
with  at  least  one  other  instructional  approach,  2)  had  predefined,  objectively  measured 
instructional  objectives,  and  3)  objectively  measured  learning  outcomes.  He  divided  the  studies 
into  those  that  concentrated  on  results  (substantive  evaluations)  and  those  that  examined  the 
effects  of  how  the  game  was  used  (procedural  studies). 

From  his  review,  Wolfe  concluded  that  there  is  evidence  that  computer-based  general 
management  games  are  effective  in  producing  knowledge  gains.  He  found  that  the  only 
alternative  instructional  approach  that  was  compared  to  games  was  case  studies.  Both  methods 
produced  learning,  but  games  appeared  to  be  superior.  It  should  be  remembered,  as  discussed 
above,  games  can  be  designed  as  hybrids,  incorporating  the  characteristics  of  case  studies. 
Although  he  reported  no  data  on  these  game-case  study  hybrids,  in  future  studies,  this  approach 
may  be  found  to  be  effective. 

Wolfe  found  less  clear  data  on  the  way  games  are  used.  Some  studies  found  less  complex 
games  as  effective  as  more  complex  games  while  others  found  that  more  complex  games 
provided  better  learning  outcomes.  However,  this  result  is  confounded  in  at  least  one  study 
because  the  more  complex  game  included  more  relevant  topics  than  the  less  complex  games. 
Wolfe  found  little  data  on  the  role  of  the  instructor  in  influencing  the  learning  outcomes  of 
games  other  than  that  some  type  of  instructor  involvement  (instructional  support)  appeared  to  be 
required.  Instructional  support  is  one  of  the  topics  discussed  in  the  next  section. 

OTHER  ISSUES  IN  INSTRUCTIONAL  GAMING 

Types  of  Competition  in  a  Game 

As  noted  above,  competition  (challenge)  is  one  of  the  most  important  characteristics  that 
make  games  enjoyable  (Csikszentmihalyi,  1990)  and  motivating  (Malone,  1981;  Malone  & 
Lepper,  1987).  Fisher  (1976)  examined  three  different  types  of  motivation  in  a  vocabulary  skills 
game  (The  Dictionary  Game).  Three  groups  of  college  students  played  the  game  under  different 
types  of  competition  conditions  and  their  performance  was  compared  with  each  other  and  with  a 
control  group  that  did  not  play  the  game.  The  first  treatment  group  played  the  game  with  1  -to- 1 
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person  interpersonal  competition.  The  second  treatment  group  played  with  l-to-2  person 
interpersonal  competition.  The  third  treatment  group  played  as  three-person  groups  with  1-to-l 
intergroup  competition.  Post-game  performance  was  measured  on  a  ten-item  multiple-choice 
test.  Results  showed  that  learners  in  all  three  game  conditions  performed  better  than  the  non¬ 
game  group.  The  l-to-2  interpersonal  competition  group  had  the  highest  score,  followed  by  the 
1-to-l  interpersonal  competition  group  and  the  1-to-l  intergroup  competition  group.  These 
results  show  that  specific  types  of  competition  can  affect  the  instructional  effectiveness  of  a 
game.  However,  since  the  game  was  not  compared  to  any  alternate  types  of  instruction,  it  is  not 
possible  to  say  that  gaming  is  the  preferred  approach  to  teach  vocabulary  skills. 

Adding  Instructional  Support  to  a  Game 

Leutner  (1993)  investigated  the  effectiveness  of  two  varieties  of  instructional  support  in  a 
computer-based  simulation  game  to  teach  economic  concepts  to  seventh-graders  and  later  to 
university  students.  The  game,  Hunger  in  the  Sahel,  is  a  role-playing  simulation  where  learners 
run  a  small  farm  and  must  deal  with  issues  such  as  rainfall  and  temperature  extremes.  The  two 
varieties  of  instructional  support  were  system-initiated  advice  and  learner-requested  background 
information.  He  found  that  learners  without  any  instructional  support  learned  to  play  the  game, 
but  only  learned  a  minimal  amount  about  domain-specific  concepts.  The  opposite  occurred  with 
the  learners  given  advice.  They  learned  more  domain-specific  concepts,  but  only  learned  to  play 
the  game  to  a  limited  degree.  These  results  indicate  that  instructional  support  is  essential  if  the 
goal  of  the  game  is  to  provide  instruction.  Leutner  (1993)  stated  that  it  is  “very  useful  to  make 
pieces  of  information,  which  are  implicitly  available  in  the  system,  explicit  through  appropriate 
instructional  support  during  system  exploration”  (p.  219). 

The  importance  of  instructional  support  is  also  discussed  by  de  Jong  and  van  Joolingen 
(1998).  These  authors  review  the  effectiveness  of  discovery  learning  in  simulated  environments. 
Discovery  learning  is  often  cited  as  one  of  the  strengths  of  instructional  simulations  and  games. 
The  assumption  is  that  learners  gain  a  deeper  understanding  of  a  topic  if  they  are  allowed  to 
freely  explore  a  domain  and  “discover”  important  information  and  interactions  among 
phenomena.  However,  after  reviewing  many  studies  on  discovery  learning  in  simulations,  these 
authors  stated:  “The  general  conclusion  that  emerges  from  these  studies  is  that  there  is  no  clear 
and  univocal  outcome  in  favor  of  simulations”  (de  Jong  and  van  Joolingen,  1998,  p.  181).  They 
explain  these  results  by  highlighting  several  intrinsic  problems  with  discovery  learning.  These 
problems  center  on  difficulties  that  learners  have  with  forming  and  testing  hypotheses  in  these 
discovery  environments.  The  reader  is  encouraged  to  consult  the  original  source  for  details  on 
these  problems.  The  authors  conclude  that  the  crucial  aspects  of  successful  discovery  learning 
are  well  designed  instructional  goals  and  instructional  support  for  the  learner. 

Mayer,  Mautone,  and  Prothero  (2002)  investigated  the  effect  of  adding  prior  pictorial 
representations  of  possible  features  that  would  be  encountered  in  a  game  to  teach  geology 
principles.  These  researchers  reported  the  results  of  4  experiments  with  college  students.  A 
game  called  The  Profile  Game  was  designed  to  help  learners  determine  which  geological 
features  were  present  on  a  certain  portion  of  the  planet’s  surface.  Prior  to  playing  the  game  in 
each  experiment,  the  students  either  received  prior  pictorial  support  or  did  not.  The  specifics  of 
how  the  support  was  used  differed  across  the  experiments.  In  the  first  two  experiments,  no 
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differences  were  found  due  to  the  pictorial  support.  However,  in  the  second  two  experiments, 
the  pictorial  support  groups  outperformed  the  no-support  groups.  The  support  given  in  the 
second  two  experiments  was  accompanied  with  more  detailed  instructions  on  how  to  use  the 
support.  Thus,  it  appears  that  instructional  support  can  enhance  the  instructional  effectiveness  of 
games  if  it  supports  “guided  discovery”  rather  than  just  providing  information. 

What  is  Learned  in  an  Adventure  Game? 

Adventure  games  are  very  popular  and  many  have  advocated  their  use  for  instructional 
purposes.  To  determine  what  is  learned  in  this  type  of  game,  Ju  and  Wagner  (1997)  had  12 
college  students  play  Indiana  Jones  and  the  Fate  of  Atlantis  in  which  they  had  to  find  their  way 
through  a  South  American  jungle.  The  students  played  the  game  individually.  After  an  hour  of 
play,  they  stopped  the  game  and  asked  the  players  to  answer  a  set  of  questions,  referring  to  the 
material  encountered  during  previous  play.  No  player  answered  all  questions  correctly.  When 
the  responses  were  analyzed  by  question  type,  they  found  that  more  correct  answers  were  given 
to  the  questions  that  asked  about  physical  objects  in  the  game  or  attributes  about  the  game  (mean 
correct  responses  of  .83  and  .84  respectively).  Questions  about  concepts  and  cause-effect 
thinking  only  produced  mean  correct  responses  of  .55  and  .50.  The  least  successful  answers  (.38 
correct)  were  on  questions  that  required  the  user  to  respond  with  a  “plan”  or  procedure.  These 
results  show  that  different  types  of  learning  can  result  from  an  adventure  game.  This  is  a  strong 
indication  of  the  importance  of  designing  instructional  games  to  contain  events  that  meet  specific 
instructional  objectives. 

The  Danger  of  Evaluator  Interest 

An  area  of  game  research  that  indicates  how  the  evaluator’s  “interests”  may  influence  the 
results  is  the  investigation  of  chess  playing  as  a  way  to  develop  children’s  cognitive  skills  and 
scholastic  achievement.  Dauvergn  (2000)  summarized  several  studies  that  he  claimed  show  that 
“chess  is  one  of  the  most  powerful  educational  tools  available  to  strengthen  a  child’s  mind”  (p.  2 
of  6).  However,  as  Thompson  (2003)  pointed  out,  most  of  these  studies  were  quasi-experimental 
in  nature  and  were  conducted  by  chess  enthusiasts  (some  from  the  U.  S.  Chess  Federation). 
Thompson  (2003)  conducted  a  controlled  study  that  compared  the  scholastic  performance  of 
chess  players  and  non-chess  players.  When  he  controlled  for  grade  level  and  IQ,  he  found  no 
significant  effect  of  chess  playing  on  scholastic  performance.  He  concluded  that  the  chess 
enthusiasts  want  to  see  a  positive  effect  from  playing  chess  and  therefore  do  not  control  for  other 
factors  like  general  intelligence.  His  belief  is  that  children  who  are  interested  in  chess  also  tend 
to  be  more  capable  in  general  scholastic  skills.  Whichever  view  is  correct,  it  is  important  that 
the  interests  and  goals  of  researchers  be  accounted  for  when  interpreting  the  results  of  their 
investigations. 


The  Instructional  Value  of  a  Game  Developed  for  Another  Purpose 

Recently,  a  game  developed  for  Army  recruiting  was  evaluated  to  determine  its  instructional 
effectiveness.  The  game,  America ’s  Army,  was  developed  by  the  Office  of  Economic  and 
Manpower  Analysis  at  the  United  States  Military  Academy  as  a  recruiting  tool.  It  was  intended 
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to  inform  the  “recruiting  age”  public  about  the  U.  S.  Army.  Because  it  was  so  popular  (over  2 
million  players  registered  on  the  web  site),  some  believed  it  could  be  assessed  to  identify 
features  that  motivated  players  and  provided  instruction. 

Belanich,  Sibley,  and  Orvis  (2004)  assessed  the  knowledge  that  21  players  gained  from  the 
game  using  a  35-question  posttest.  The  questions  were  designed  to  assess  procedural,  episodic, 
and  factual  knowledge  that  was  either  relevant  to  game  play  or  that  did  not  impact  game  play. 
They  found  that  players  correctly  answered  a  higher  percentage  of  procedural  questions, 
followed  by  episodic  and  factual  questions  in  descending  order.  They  also  found  that  the 
realism,  challenge,  exploration,  and  control  afforded  by  the  game  were  the  factors  that 
influenced  player  motivation. 

On  the  surface,  these  results  seem  to  indicate  that  the  game  provided  effective  instruction. 
However,  when  one  examines  the  study  more  closely,  it  becomes  apparent  that  the  players  only 
learned  procedures,  experiences,  and  facts  about  the  game.  There  is  no  indication  that  any  of  the 
learned  information  is  relevant  to  real-world  Army  requirements.  For  example,  one  of  the 
procedural  questions  asks  how  to  select  a  “Flash/Bang”  grenade.  The  possible  answers  involve 
pressing  a  number  key  or  clicking  the  mouse.  These  procedures  are  irrelevant  in  the  real-world. 
From  reading  the  report,  it  does  not  seem  that  the  game  provided  real-world  relevant  instruction, 
such  as  when  to  choose  this  type  of  grenade. 

This  study  illustrates  one  of  the  greatest  dangers  of  using  games  for  instruction.  Yes,  they 
may  be  motivational  and  they  may  help  learners  retain  information.  However,  if  the  game  is  not 
designed  to  directly  support  specific  instructional  objectives  related  to  actual  job  requirements, 
much  of  the  learning  may  be  irrelevant.  This  is  also  what  was  found  in  the  evaluation  of  The 
Oregon  Trail  Game  (Caftori,  1994).  As  discussed  earlier,  children  only  learned  “shoot-em-up” 
skills  rather  than  the  historical  information  that  game  designers  hoped  the  game  would  provide. 

This  study  (Belanich,  et  al.,  2004),  like  many  others,  indicates  the  potential  of  games  for 
instruction.  However,  it  also  highlights  the  need  to  carefully  design  and  develop  games  that  will 
support  specific  instructional  objectives  and  that  will  fit  into  larger  programs  of  instruction. 

The  Importance  of  Debriefing 

After  participation  in  an  instructional  game,  it  is  very  important  to  conduct  a  debriefing 
session  with  the  participants  (Crookall,  1992;  Garris,  et  al.,  2002).  Debriefing  gives  the  learners 
the  opportunity  to  reflect  on  their  experience  with  the  game  and  understand  how  this  experience 
supported  the  instructional  objectives  of  the  course  or  program  of  instruction.  Lederman  (1992) 
reviewed  the  literature  on  debriefing  and  presented  a  model  of  the  debriefing  process. 

According  to  her  model,  debriefing  should  consist  of  three  phases: 

•  Phase  1:  Systematic  reflection  and  analysis.  This  phase  introduces  the  learners  to  a 
systematic  self-reflective  process  about  their  game  experience.  It  should  include  a  summary  of 
the  game  experience  as  it  supports  the  instructional  objectives. 

•  Phase  2:  Intensification  and  personalization.  The  second  phase  is  a  deeper  examination  of 
the  learners’  game  experiences.  Each  learner  should  be  guided  through  reflections  on  their 
own  individual  experiences  and  the  meanings  these  experiences  have  for  them.  It  is  the  job  of 
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the  debriefing  facilitator  (e.g.,  instructor)  to  help  ensure  that  the  meaning  of  the  experiences  tie 
into  the  instructional  objectives. 

•  Phase  3:  Generalization  and  application.  Here,  the  learners  are  led  beyond  their  own 
individual  experiences  to  the  broader  applications  of  the  experiences.  For  example,  they  might 
engage  in  discussions  of  how  their  experiences  can  contribute  to  their  success  on  the  job  or  in 
additional  learning  situations. 

The  key  point  is  that  no  instructional  game  should  be  conceived  as  a  stand-alone  activity.  It 
should  be  designed  and  implemented  as  part  of  a  larger  instructional  program  that  meets  the 
specific  instructional  requirements  of  the  task  and  the  learners.  As  Lederman  puts  it,  “The 
process  of  debriefing  is  not  ancillary  to  the  educational  experience  to  which  it  is  tied.  Debriefing 
is  an  integral  part  of  any  learning  experience  that  is  designed  to  be  experience  based” 

(Lederman,  1992,  p.  158). 
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SUMMARY  OF  EMPIRICAL  EVIDENCE  ON  INSTRUCTIONAL  GAMES 

Instructional  games  have  been  used  for  a  wide  variety  of  tasks  with  learners  of  different  ages. 
Past  reviews  have  found  that  the  majority  of  articles  on  instructional  games  are  opinions  about 
whether  to  use  games  and  about  the  potential  of  games  to  provide  effective  instruction  (e.g., 
Dempsey,  et  al.,  1993-1994).  Far  fewer  articles  provided  empirical  data  on  research  that  has 
investigated  the  instructional  effectiveness  of  games.  This  was  also  found  in  the  present  review. 
Over  270  documents  were  obtained  from  the  literature  on  games.  Of  these,  only  48  provided 
empirical  data  on  the  instructional  effectiveness  of  games.  This  review  discussed  the  results  of 
other  review  articles  and  also  articles  on  the  effectiveness  of  specific  games.  Table  4  provides  a 
summary  of  the  48  empirical  articles  discussed  in  earlier  sections. 

Several  observations  can  be  derived  from  an  examination  of  Table  4.  First,  the  empirical 
research  on  instructional  games  examined  their  effects  for  a  wide  range  of  age  groups.  Twelve 
of  the  studies  examined  the  learning  effects  of  games  with  elementary  school  children  and  two 
with  high  school  students.  Twenty-one  studies  used  instructional  games  with  college  students. 
Six  studies  examined  game  effects  with  military  trainees  and  the  remainder  of  the  studies 
evaluated  games  for  adults  working  in  various  industries.  The  results  of  any  given  study  must  be 
evaluated  in  a  restricted  manner.  We  cannot  necessarily  generalize  the  results  of  a  study 
conducted  with  one  age  group  to  another  age  group. 

The  empirical  research  also  includes  studies  that  examined  the  instructional  effectiveness  of 
games  for  many  different  tasks.  These  tasks  range  from  basic  mathematics  principles  to  complex 
business  and  marketing  decision  making.  They  include  general  reasoning  skills  and  specific 
electronics  repair  tasks.  Care  should  also  be  taken  when  generalizing  from  one  instructional  task 
to  tasks  in  another  domain. 

The  empirical  research  does  not  make  a  compelling  case  for  games  as  the  preferred 
instructional  method.  In  most  cases,  the  research  shows  no  instructional  advantage  of  games 
over  other  instructional  approaches  (such  as  lectures).  In  several  cases,  games  were  shown  to 
provide  effective  learning,  but  were  not  compared  to  other  instructional  methods.  The  research 
does  not  allow  us  to  conclude  that  games  are  more  effective  than  other  well-designed 
instructional  activities. 

Too  much  of  the  empirical  research  on  instructional  games  contains  methodological  problems 
(e.g.,  experimental  confounds)  that  make  it  difficult  to  draw  valid  conclusions  about  the 
effectiveness  of  the  games.  Researchers  need  to  ensure  that  they  understand  experimental  design 
and  apply  sound  decisions  when  designing  and  reporting  their  research.  In  addition,  editors  of 
educationally  oriented  journals  need  to  filter  out  studies  that  do  not  follow  sound  experimental 
design  procedures. 
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Table  4: 

Summary  of  Empirical  Effectiveness  Studies  Reviewed 


Study 

■H9 1 

Results 

Koran  &  McLaughlin  (1990) 

Math  Facts 

5th  Gr. 

No  Differences 

Rowe  (2001) 

Math 

Elementary  School 

Only  one  of  two  games 
provided  learning.  No 
comparison  to  other  approaches. 

VanEck  &  Dempsey  (2002) 

No  Differences 

Probability 

4-5  yrs. 

Can’t  Determine 

Math  Skills 

Pre-K-lst  Gr. 

Can’t  Determine 

Rosas,  et  al.  (2003) 

Basic  Math  & 

Reading 

1st  &  2nd  Gr. 

No  Differences 

Malouf  (1987-88) 

Vocabulary 

6-8th  Gr. 

No  Differences 

Renaud  &  Stolovitch  (1988) 

Pedestrian  Safety 

5  yrs. 

Learning,  but  no  comparison 

Wishart  (1990) 

Motivation 

5-12  yrs. 

Learning,  but  no  comparison 

Costable,  et  al.  (2003) 

Basic  Logic 

nil  trayii&iBiiiii 

No  Differences 

Wiebe  &  Martin  (1994) 

5th-6,h  Gr. 

No  Differences 

Virvou,  et  al.  (2005) 

Geography 

4th  Gr. 

Game  favored  over  non-game 
software 

Ronen  &  Eliahu  (2000) 

Electronics 

15  yrs. 

Slight,  but  not  effective  for 
certain  groups 

White  (1984) 

H.  School 

Learning,  but  no  comparison 

Thomas,  et  al.  (1997) 

Self-efficacy  in 
HIV/AIDS  prevention 

Adolescents 

Can’t  Determine 

Diabetes  Facts 

8-16  yrs. 

Can’t  Determine 

Serrano  &  Anderson  (2004) 

Nutrition  Facts 

5th  Gr. 

Learning,  but  no  comparison 

Szafron  &  Mandolini  (1980) 

Sociology 

College 

Learning,  but  no  comparison 

Psychology 

College 

No  Differences 

Fraas  (1982) 

Economics 

College 

No  Differences 

(game  better  for  learners  with 

less  previous  knowledge) 

Gremmen  &  Potters  (1995) 

Economics 

College 

Game  supplementing  lecture 
favored  over  lecture  alone 

Wood  &  Stewart  (1987) 

Practical  Reasoning 

College 

Learning,  but  only  compared  to 
no  instruction. 

No  Differences 

Williams  (1980) 

Attitudes  toward 
subject  matter 

College 

Favored  Game  over  non- 
related  activity 

Bredemeier,  et  al.  (1982) 

Attitudes  toward 
subject  matter 

College 

Game  successful  in  appropriate 
context 

Barak,  et  al.  (1987) 

Empathy 

College 

Favored  Game  over  lecture  & 

l  raw.'1!  c  ~  iii 


discussion. _ 

Game  interfered  with  learnini 
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Table  4:  (Continued) 


&f5we?fTfrS 

Results 

Miller,  et  al.  (1999) 

Electrical  Interactions 

College 

No  comparison  to  alternate  instruction. 
Different  versions  of  game  were  compared. 

Crown  (2001) 

Engineering 

Visualization 

College 

Can’t  Determine 

Shrestha  (1990) 

Underwater  Sound 
Propagation 

College 

No  Differences 

Westbrook  & 
Braithwaite  (2001) 

Health  Care  Decisions 

College 

Can’t  Determine 

Marketing 

Game  alone  as  good  as  game  and  lecture 

IBHUHIIHH 

Team  Cohesion  & 
Attitudes 

College 

No  Comparison 

Marketing 

College 

No  sutmort  for  game 

Gopher,  et  al.  (1994) 

Attention  Allocation 

Skills 

Pilot  Trainees 

Learning,  but  no  comnarison 

Garris,  et  al.  (2002) 

Periscope  Skills 

Navy 

Trainees 

Favored  Game  (over  trainer  without  game 
elements) 

Ricci,  et  al.  (1996) 

CBRD  Defense 

Principles 

Navy 

Trainees 
(Mean  age  = 
20yrs.) 

No  Differences  (“game”  slightly  better  for 
retention) 

Oxford,  et  al.  (1987) 

Technical  Skills  & 
Terminology 

Army 

Trainees 

Favored  Game  (over  workbook) 

Whitehill  & 

McDonald  (1993) 

Electrical  Repairs 

Navy 

Trainees 

No  Differences 

Electronics 

Fundamentals 

Navy 

Trainees 

Drill  &  Practice  favored  over  game 

Business  Issues 

Adults 

No  Comparison 

I'mtmximmn 

Attendance  Attitudes 

Adults 

No  effect 

Fisher  (1976) 

Different  types  of 
Competition 

College 

No  Comparison 

Leutner  (1993) 

Economics  Concepts 

7th  Gr.  & 
College 

Without  instructional  support,  Only 
Learned  Game 

Mayer,  et  al.  (2002) 

Geology 

College 

Instructional  support  enhanced  value  of 
game 

Ju  &  Wagner  (1997) 

College 

Learned  more  about  physical  objects  in 
game  than  concepts 

Thompson  (2003) 

Effect  of  playing  chess 
on  scholastic 
performance 

Elementary 

school 

children 

No  effect 

Learned  only  game-relevant  skills  not  real- 
world  skills 
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A  REEXAMINATION  OF  THE  “CLAIMS” 

As  discussed  earlier,  Greenblat  (1981)  summarized  evidence  in  support  of  six  categories  of 
“claims”  about  the  efficacy  of  games  used  for  instructional  purposes.  We  can  use  the  same 
categories  of  claims  to  help  us  summarize  the  research  on  the  instructional  uses  of  games  since 
that  time.  By  comparing  the  evidence  presented  by  Greenblat  in  1981  to  the  results  of  later 
research  we  can  draw  some  conclusions  about  the  progress  that  has  been  made  over  the  last  25 
years  in  our  understanding  of  the  effectiveness  of  games  for  instructional  purposes.  Table  5 
summarizes  the  evidence,  then  and  now. 

Comparing  the  evidence  on  the  instructional  effectiveness  of  games  that  was  available  in  1981 
to  the  results  of  more  recent  research  shows  that  there  are  still  many  questions  that  need  to  be 
answered  about  when  and  how  to  use  games.  The  first  “claim,”  that  games  improve  the 
motivation  and  interest  of  learners,  is  still  weakly  supported.  There  has  been  little  additional 
evidence  beyond  a  few  studies  that  use  self-reports  or  time-on-task  to  show  that  games  are 
motivational.  Unfortunately,  there  is  little  evidence  that  these  measures  of  motivation  are  related 
to  improved  task  performance.  Games  do  motivate.  They  motivate  players  to  play  the  game. 
This  can  be  beneficial  if  the  game  is  designed  to  target  and  meet  instructional  objectives. 
Otherwise,  learners  may  spend  their  time  learning  to  be  successful  at  the  game  without  receiving 
instructional  benefits  from  these  experiences. 

The  second  “claim,”  that  games  enhance  cognitive  learning,  continues  to  be  supported.  The 
research  shows  that  people  can  learn  from  games.  However,  the  research  does  not  indicate  that 
games  are  superior  to  other  instructional  methods  in  all  cases.  Like  any  instructional  activity, 
games  should  be  chosen  because  they  provide  learners  with  interactive  experiences  that  help 
them  meet  instructional  objectives. 

The  third  and  sixth  “claims”  were  not  supported.  No  additional  evidence  was  found  that 
games  change  later  course  work.  This  effect  may  occur,  but  no  research  was  found  to  support 
this  “claim.”  This  may  be  because  it  is  difficult  to  track  learners  from  one  class  to  another  and 
document  this  effect.  Likewise,  no  evidence  was  found  for  the  sixth  “claim,”  that  the  use  of 
games  changes  the  classroom  structure  and  relations.  This  does  not  mean  that  instructors  who 
use  games  in  their  courses  do  not  change  their  classroom  structure  or  that  game  players  do  not 
change  how  they  relate  to  other  learners.  Some  anecdotal  evidence  indicates  that  these  changes 
do  occur,  but  no  empirical  evidence  was  found. 

Finally,  the  fourth  and  fifth  “claims,”  that  games  change  learners’  feelings  (affect)  about  the 
learning  domain  and  learning  in  general  has  mixed  support.  Two  studies  (Williams,  1980;  and 
Bredemeier,  et  al.,  1982)  provide  some  support  for  the  utility  of  games  to  change  learners’ 
attitudes  toward  subject  matter  (claim  4).  No  empirical  research  was  found  that  supports  the 
claim  that  games  can  change  learners’  attitudes  about  learning  in  general  (claim  5). 
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Table  5: 

Evidence  in  Support  of  the  “Claims”: 
Then  and  Now 


Categories  of 
Claims 

Original  Evidence 
(prior  to  1981) 

Later  Evidence 
(1981  -  2005) 

1.  Motivation  & 
Interest 

Strongest  support.  A  great  deal  of 
anecdotal  reports.  Only  one  study 
(Robinson  et  al.,  1966)  used  several 
indicators  of  motivation  to  show 
simulation-games  generated  greater 
interest  than  other  modes  of  teaching. 

Little  additional  evidence.  A  few  studies 
found  that  learners  indicated  that  they 
enjoyed  games  and  spent  more  time 
playing.  However,  only  weak 
connection  between  this  and  improved 
performance. 

2.  Cognitive  Learning 

Some  weak  emoirical  evidence 
favoring  games.  Some  showing  no 
differences. 

Similar  pattern.  Some  studies  show  that 
games  are  effective  for  some  learning 
tasks,  but  do  not  show  them  superior  to 
other  instructional  approaches.  Some 
evidence  that  games  can  be  detrimental 
to  learning  if  thev  do  not  include 
instructional  support.  Games  have  been 
shown  to  be  more  effective  if  they  are 
followed  bv  a  debriefing  session  that 
highlights  the  importance  of  the  game 
experiences  in  terms  of  instructional 
objectives. 

3.  Changes  in  Later 
Course  Work 

None 

None  Found 

4.  Affective  Learning 
(re:  Subject  Matter) 

Mixed  results.  Some  anecdotal 
evidence.  Empirical  evidence  shows 
increases  in  both  positive  and  in 
negative  attitudes. 

Two  studies  provide  some  additional 
support.  There  are  indications  that  a 
game  is  more  effective  if  used  in  the 
appropriate  context. 

5.  General  Affective 
Learning 

Almost  none 

None  Found 

6.  Changes  in 
Classroom  Structure 
&  Relations 

None 

None  Found 

In  summary,  this  review  identified  several  general  problems  with  instructional  game  research 
that  make  it  difficult  to  draw  firm  conclusions.  These  problems  include: 

•  There  are  many  published  articles,  but  most  are  opinions  about  the  “potential”  of  games. 
There  are  few  empirical  studies. 

•  There  is  considerable  confusion  in  the  use  of  terms.  When  is  a  simulation  a  game?  Are 
all  competitive  activities  games?  It  is  often  difficult  to  determine  if  the  instructional 
activity  was  actually  a  game. 

•  There  are  many  methodological  flaws  in  the  empirical  game  research,  such  as  poor 
experimental  designs  that  lack  the  control  of  possible  confounding  variables  (e.g.,  events 
prior  to  and  during  the  evaluation). 
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•  Some  of  the  research  appears  to  be  biased  by  the  evaluators’  interest  in  “proving”  the 
effectiveness  of  their  game  (i.e.,  they  developed  the  game  and  wanted  it  to  work). 

•  Too  much  research  fails  to  use  control  groups  who  experienced  an  alternate  form  of 
instruction  to  compare  to  the  game. 

•  Most  published  articles  provide  no  clear  and  detailed  description  of  the  game  itself  (e.g., 
what  characteristics  made  it  a  “game”). 


TO  GAME  OR  NOT  TO  GAME? 

As  can  be  seen  from  the  above  discussions,  the  research  on  the  use  of  instructional  games  is 
mixed.  It  appears  that  games  can  be  of  instructional  value  if  they  are  well  designed  and  targeted 
to  meet  specific  instructional  objectives.  Unfortunately,  many  program  managers  and  game 
developers  do  not  appreciate  the  importance  of  instructional  design.  They  often  assume  that  the 
game  is  sufficient,  in  itself,  to  provide  the  necessary  instruction.  Squire  (2005)  conducted  case 
studies  of  three  companies  that  develop  game-based  learning  products.  “It  is  worth  noting  that 
none  of  the  featured  companies  started  in  instructional  design... they  come  from  business 
strategy,  marketing,  and  the  games  industry”  (p.  13,  my  emphasis).  Although  each  company 
used  interdisciplinary  design  teams  to  create  their  instructional  games,  none  of  the  teams 
included  instructional  developers.  The  teams  usually  consisted  of:  1)  graphic  artists,  2)  program 
managers,  and  3)  programmers.  Commenting  on  the  avoidance  of  instructional  designers,  Squire 
stated,  “Most  game-based  learning  approaches  do  not  employ  that  particular  category  of  expert 
whatsoever”  (p.  35).  In  most  cases,  the  game  designers  fulfilled  the  role  of  instructional 
developer.  It  appears  that  the  “instructional  gaming”  industry  does  not  value  the  skills  of 
instructional  developers. 

This  is  not  a  new  phenomenon.  In  the  late  1980s,  Greenblat  (1987)  observed  that  the 
teaching  enterprise  was  undervalued  in  our  society.  Technological  issues  involved  in  game 
development  seem  better  able  to  catch  people’s  interest,  while  the  development  of  instructional 
objectives  and  logical  programs  of  instruction  seems  to  be  boring.  The  data  on  instructional 
effectiveness  of  games  (e.g.,  Randel,  et  al.,  1992;  Parchman,  et  ah,  2000;  Wolfe,  1997)  indicate 
that  the  role  of  the  instructor  and  the  way  a  game  is  incorporated  into  an  instructional  program 
are  major  factors  in  whether  a  game  will  contribute  to  successful  learning.  Nevertheless,  it  is  not 
clear  that  current  game-development  teams  understand  the  principles  of  instructional 
development.  It  may  be  up  to  the  program  managers  and  other  individuals  who  procure 
instructional  games  to  demand  that  they  support  instructional  objectives  and  that  this  support  be 
demonstrated.  This  will  then  probably  require  that  game  companies  begin  to  include  individuals 
who  understand  learning  and  instruction  on  their  development  teams.  As  illustrated  in  Figure  2, 
it  is  more  likely  that  games  will  be  instructionally  effective  if  the  specific  characteristics  of  the 
game  (e.g.,  setting,  player  roles  and  activities,  rules,  etc.)  overlap  with  specific  instructional 
objectives.  This  overlap  must  be  consciously  structured  on  the  basis  of  a  thorough  analysis  of 
the  reasons  for  the  instruction  and  the  instructional  objectives  to  be  met. 
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Figure  2: 

Instructional  Effectiveness  as  Degree  of  Overlap 
among  Instructional  Objectives  and  Game  Characteristics 


Rieber  (1996)  advocated  a  mixing  of  simulations  and  games  in  a  hybrid  learning 
environment.  This  approach  would  use  the  strengths  of  simulations  (e.g.,  dynamic  and 
interactive  representations  of  real-world  systems  or  phenomena)  and  the  strengths  of  games  (e.g., 
challenge  and  fantasy)  to  create  instructionally  sound,  realistic  learning  activities.  Additional 
research  needs  to  be  conducted  that  will  provide  guidance  about  how  to  create  effective  hybrid 
learning  environments  (e.g.,  how  to  determine  if  the  specific  features  of  specific  hybrid  games 
will  support  instructional  objectives). 

Much  of  the  empirical  research  indicates  that  instructional  games  are  only  effective  if  they  are 
designed  to  support  instructional  objectives.  This  is  more  likely  if  the  design  of  the  instructional 
game  is  the  result  of  a  systematic  analysis  process.  Atkinson  (1977)  advocated  that  the  design 
and  development  of  instructional  games  should  follow  the  same  basic  systems  approach  that  is 
required  for  the  design  of  any  instructional  activity.  The  basic  steps  in  this  approach  are  shown 
in  Figure  3  as  applied  to  the  design  and  use  of  instructional  games.  The  first  section  of  the 
approach  includes  steps  that  are  exactly  the  same,  no  matter  what  the  outcome  (e.g.,  choosing  a 
game  or  another  instructional  activity).  The  first  two  steps  are  to  identify  the  instructional 
problem  and  the  instructional  objectives.  A  problem  statement  is  a  general  statement  of  the 
overall  instructional  problem  (e.g.,  improve  students  overall  understanding  of  and  competence  in 
a  topic  area).  Next,  specific  instructional  objectives  must  be  determined.  These  objectives 
should  be  stated  in  terms  of  observable  learner  behaviors,  under  specific  conditions  that  help  the 
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SECTION  1:  Understanding  the  Instructional  Environment 


Figure  3: 

A  Systematic  Approach  to  Instructional  Game  Design  and  Use 


learners  reach  an  acceptable  level  of  performance.  These  foundational  requirements  are  next 
used  to  help  determine  the  possible  alternative  instructional  strategies  and/or  approaches  that  can 
be  selected.  The  choice  of  a  specific  instructional  strategy  is  then  determined  on  the  basis  of  the 
constraints  of  each  instructional  situation  (e.g.,  numbers  of  students,  available  facilities,  budget, 
etc.).  Only  when  a  gaming  approach  has  been  selected  from  the  alternative  instructional 
strategies  does  one  move  to  the  next  section:  the  design  or  modification  of  a  specific  game. 

The  second  section  of  Figure  3  shows  the  basic  steps  in  developing  an  instructional  game. 

The  fourth  step  is  to  develop  the  game  model.  Like  any  model,  it  is  a  simplified  representation 
of  only  certain  elements  of  reality.  The  specific  elements  chosen  for  the  game  should  be  selected 
on  the  basis  of  how  they  will  support  the  instructional  objectives  already  identified.  Step  5  is  the 
identification  of  how  the  students  will  participate  in  the  game.  Several  questions,  like  the 
following,  must  be  answered  to  support  game  design.  Will  each  student  play  the  same  role  or 
different  roles?  Will  they  play  as  individuals  or  as  a  team?  Will  learner  actions  affect  the 
outcome  of  the  game  or  do  they  just  observe?  These  and  other  questions  will  lead  to  the 
development  of  specific  rules  and  events  in  the  game.  Finally,  the  criteria  for  winning  the  game 
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must  be  developed.  Winning  the  game  should  involve  improved  learner  performance  on  the 
instructional  objectives.  Sometimes  an  existing  game  can  be  modified  for  a  new  instructional 
situation.  Even  if  an  existing  game  is  chosen,  it  must  be  modified  to  meet  the  specific 
requirements  of  the  new  instructional  situation. 

The  third  section  of  Figure  3  shows  the  three  main  steps  involved  in  the  implementation  and 
evaluation  of  an  instructional  game.  In  addition  to  the  game,  supporting  materials,  like  players’ 
instructions  and  teachers’  manuals  must  be  developed.  As  indicated  by  the  data  from  studies 
such  as  Leutner  (1993)  and  Mayer,  et  al.  (2002),  instructional  support  that  explains  the  purpose 
of  the  game  and  how  to  play  it  can  enhance  the  instructional  value  of  the  game.  In  step  8,  the 
instructional  game  is  evaluated.  Ideally  this  evaluation  should  compare  the  effectiveness  of  the 
game  (as  measured  by  learner  performance)  to  the  effectiveness  of  alternate  instructional 
approaches.  The  final  step  in  this  process  uses  the  results  of  the  evaluation  to  modify  the  game 
to  improve  its  instructional  effectiveness. 

In  any  systematic  process,  like  the  one  depicted  in  Figure  3,  it  is  necessary  to  make  specific 
design  decisions  about  the  game.  Malone  (1981, 1982)  reported  the  results  of  a  series  of  studies 
that  detailed  the  characteristics  that  made  several  simple  video  games  enjoyable  to  students.  As 
discussed  earlier,  he  used  his  results  to  develop  a  general  framework  for  analyzing  the  appeal  of 
instructional  environments,  computer  games,  and  computer  interfaces  (see  Table  1).  The 
framework  was  based  on  three  categories:  challenge,  fantasy,  and  curiosity.  Because  this 
framework  is  consistent  with  the  important  characteristics  of  games  and  motivational 
instructional  environments  summarized  earlier  (e.g.,  Malone  &  Lepper,  1987;  Lepper  &  Malone, 
1987;  Csikszentmihalyi,  1990;  Leemkuil,  et  al.,  2000),  it  has  been  adapted  to  help  us  make 
recommendations  about  the  design  of  effective  and  motivational  instructional  games.  Table  6 
shows  these  design  recommendations  applied  specifically  to  instructional  games. 

One  thing  that  is  fairly  clear  is  that  instructional  games  are  not  a  panacea.  Like  any 
instructional  medium  or  approach,  games  must  provide  a  means  for  learners  to  engage  in 
cognitive  and/or  motor  interactions  that  directly  support  instructional  objectives.  Gratuitous  use 
of  games  or  the  use  of  games  with  no  clear  instructional  goals  will  probably  increase  the  cost  of 
the  instruction  without  providing  the  instructional  benefit  that  learners  require  (see  the  earlier 
discussion  of  de  Jong  &  van  Joolingen,  1998).  If  a  game  is  chosen  as  an  instructional  approach, 
it  is  important  that  it  includes  instructional  support,  such  as  emphasizing  important  information 
in  the  game  that  support  the  instructional  objectives  (e.g.,  Leutner,  1993). 

If  designed  with  instructional  goals  in  mind,  games  can  provide  effective  instruction.  Jacobs 
and  Dempsey  (1993)  expressed  the  opinion  that  “By  facilitating  learner  involvement  via 
simulation  and  gaming  and  incorporating  sound  instructional  features,  learning  outcomes  should 
show  improvements  relative  to  other  training  methods  that  are  less  engaging  or  that  provide  less 
effective  means  of  instructional  interaction”  (p.  198).  They  suggest  that  instructional  gaming 
design  should  take  several  factors  into  account.  Instructional  games  should  include  relevant 
activities  that  make  up  the  job  or  domain  of  interest.  The  relative  “criticality”  of  the  various 
activities  should  be  determined  and  the  most  critical  should  be  included  in  the  game.  Finally,  the 
performance  of  learners  in  the  game  activities  should  be  evaluated  so  learners  are  rewarded  for 
performing  correctly  and  given  remediation  to  improve  incorrect  performance. 
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Table  6: 

Design  Recommendations  for  Instructional  Games 


Recommendations 

Challenge 

A.  Goal. 

•  There  should  be  a  clear  goal  to  the  game. 

•  The  goal  should  be  consistent  with  the  instructional  objective. 

•  Performance  feedback  should  be  provided  about  how  close  the  learner  is  to 
achieving  the  goal. 

B.  Uncertain  Outcome. 

•  The  outcome  of  reaching  the  goal  should  be  uncertain.  The  learner  should 
have  to  exert  effort  to  achieve  the  goal. 

•  If  the  subject  matter  is  complex,  the  game  should  include  multiple  levels  of 
difficulty  or  complexity.  Sub-goals  should  help  the  learner  successfully 
complete  enabling  instructional  objectives. 

C.  Competition. 

•  A  game  can  be  made  more  challenging  by  introducing  competition. 

•  Competition  can  be  against  a  live  opponent,  against  a  computer-controlled 
opponent,  or  against  a  criterion  score.  Achieving  a  criterion  score  can  be  one 
of  the  factors  that  allows  a  learner  to  advance  to  higher  levels  of  difficulty. 

•  Game  scores  should  be  clearly  related  to  the  achievement  of  instructional 
objectives  (e.g.,  not  just  how  many  shots  are  fired  accurately  or  how  many 
opponents  are  defeated). 

Fantasy 

A.  Emotional  Appeal. 

•  The  fantasy  should  embody  emotional  appeal  for  the  learners  (e.g.,  not 
everyone  reacts  positively  to  shooting  weapons  or  conversely,  to  complex 
interpersonal  situations). 

B.  Fantasy  Metaphor. 

•  The  metaphor  used  in  the  fantasy  should  embody  physical  or  other 
characteristics  that  the  learner  already  understands. 

Curiosity 

A.  Optimal  Level  of  complexity. 

•  The  game  should  provide  an  optimal  level  of  informational  complexity  to  meet 
the  needs  of  the  learners. 

•  The  level  of  complexity  should  increase  as  learners  gain  higher  levels  of 
expertise. 

B.  Incorporation  of  Interesting  Media. 

•  The  game  should  use  interesting  audio  and  visual  effects  to  enhance  the 
fantasy  and  emphasize  instructional  content. 

•  The  game  can  incorporate  elements  of  randomness  to  avoid  boredom  for  the 
learner. 

Instructional 

Quality 

A.  Logical  Instructional  Structure. 

•  The  game  should  capitalize  on  the  learner’s  desire  to  have  “well-formed” 
knowledge  structures. 

•  Elements  of  the  game  should  build  on  and  reinforce  each  other  to  help  build 
the  learner’s  knowledge  structures. 

•  Learners  should  be  able  to  easily  see  when  their  knowledge  structures  are 
incomplete.  Feedback  during  the  game  should  be  used  to  help  learners 
complete  their  knowledge  structures. 

B.  Incorporation  into  the  Larger  Instructional  Program. 

•  The  game  should  not  “stand  alone.”  Rather,  it  should  be  part  of  a  larger 
instructional  program. 

•  Debriefing  and  feedback  should  be  provided  after  the  game  to  help  the  learner 
understand  how  the  experiences  in  the  game  supports  instructional  objectives. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

This  review  of  empirical  research  on  the  effectiveness  of  instructional  games  leads  to  the 
following  five  conclusions  and  four  recommendations.  The  conclusions  are: 

•  The  empirical  research  on  the  effectiveness  of  instructional  games  is  fragmented.  The 
literature  includes  research  on  different  tasks,  age  groups,  and  types  of  games.  The  research 
literature  is  also  filled  with  ill  defined  terms,  and  plagued  with  methodological  flaws. 

•  Although  research  has  shown  that  some  games  can  provide  effective  learning  for  a  variety  of 
learners  for  several  different  tasks  (e.g.,  math,  attitudes,  electronics,  and  economics),  this  does 
not  tell  us  whether  to  use  a  game  for  our  specific  instructional  task.  We  should  not  generalize 
from  research  on  the  effectiveness  of  one  game  in  one  learning  area  for  one  group  of  learners 
to  all  games  in  all  learning  areas  for  all  learners. 

•  There  is  no  evidence  to  indicate  that  games  are  the  preferred  instructional  method  in  all 
situations. 

•  Instructional  games  should  be  embedded  in  instructional  programs  that  include  debriefing 
and  feedback  so  the  learners  understand  what  happened  in  the  game  and  how  these  events 
support  the  instructional  objectives. 

•  Instructional  support  to  help  learners  understand  how  to  use  the  game  increases  the 
instructional  effectiveness  of  the  gaming  experience  by  allowing  learners  to  focus  on  the 
instructional  information  rather  than  the  requirements  of  the  game. 

The  following  four  recommendations  may  help  the  instructional  gaming  industry  produce 
more  instructionally  effective  games. 

•  The  decision  to  use  a  game  should  be  based  on  a  detailed  analysis  of  the  learning 
requirements  and  an  analysis  of  the  tradeoffs  among  alternate  instructional  approaches. 

•  Program  managers  and  procurement  personnel  should  insist  that  game  developers  clearly 
demonstrate  how  the  design  of  a  game  will  provide  interactive  experiences  that  support 
properly  designed  instructional  objectives  (see  for  example,  Gagne  &  Briggs,  1979;  Merrill, 
1983;  1997  for  guidance  on  the  proper  design  of  instructional  objectives). 

•  Instructors  should  view  instructional  games  as  adjuncts  and  aids  to  help  support  instructional 
objectives.  Learners  should  be  provided  with  debriefing  and  feedback  that  clearly  explains 
how  their  experiences  with  the  game  help  them  meet  these  instructional  objectives. 

•  Instructor-less  approaches  (e.g.,  web-based  instruction)  must  include  all  “instructor 
functions.”  These  include  performance  evaluation,  debriefing,  and  feedback. 

Herbert  Simon  (2000)  provides  a  succinct  prescription  for  designing  improved  instruction. 
Although  he  was  not  speaking  specifically  about  instructional  games,  his  prescription  applies 
equally  well  to  instructional  games  as  to  other  forms  of  instructional  activities.  As  Simon  puts  it, 
“A  first  step  toward  improved  instruction  is  to  examine  the  understanding  we  hope  students  will 
acquire,  the  things  they  should  be  able  to  do  with  their  knowledge.. ..Next,  we  must  design  a 
series  of  experiences  that  will  enable  students  to  learn  the  relevant  cues  in  the  situations  they 
encounter  and  to  evoke  from  memory  the  actions  that  are  effective  and  appropriate  in  specific 
situations”  (pp.  117-118). 
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